User Tools

Site Tools


projects:fsharp_workshop

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
projects:fsharp_workshop [2015/04/18 17:32]
mkucia
projects:fsharp_workshop [2015/05/02 12:45] (current)
mkucia [hello(x)]
Line 92: Line 92:
 Note that F# uses indentation to indicate block structure. Find out more [[http://​fsharpforfunandprofit.com/​posts/​fsharp-syntax/​|here]]. Note that F# uses indentation to indicate block structure. Find out more [[http://​fsharpforfunandprofit.com/​posts/​fsharp-syntax/​|here]].
  
 +===== Getting data =====
  
 +The following example loads data from webpage and stores it in CSV file.
 +
 +<​hidden>​
 +<code fsarp>
 +open System.Net
 +open System
 +open System.IO
 +
 +let myURL = @"​..."​
 +
 +let fetchUrl callback url =        ​
 +    let req    = WebRequest.Create(Uri(url)) ​
 +    use resp   = req.GetResponse() ​
 +    use stream = resp.GetResponseStream() ​
 +    use reader = new IO.StreamReader(stream) ​
 +    callback reader url
 +
 +let myCallback (reader:​IO.StreamReader) url = 
 +    let htmlSource = reader.ReadToEnd()
 +    htmlSource
 +
 +let getWebpageHTML = fetchUrl myCallback myURL
 +
 +type FirstOrLast = 
 + | GetFirst
 + | GetLast
 +
 +let getDataFromHTML (rawHTML:​String) = 
 + let split (split_point:​String) (firstOrLast:​FirstOrLast) (text:​String) = 
 + let array = text.Split([|split_point|],​ 2, StringSplitOptions.RemoveEmptyEntries)
 + match firstOrLast with
 + | GetFirst -> array.First()
 + | GetLast ​ -> array.Last()
 + rawHTML
 + |> split @"<​table id=""​t2"">"​ GetLast
 + |> split @"</​table>" ​         GetFirst
 +
 +let removeAllTags text =
 +    Regex.Replace(text,​ "<​.*?>",​ String.Empty);​
 +
 +let getCSVfromHTML rawHTML = 
 + let mutable (text:​String) = String.Empty
 + text <- rawHTML
 + text <- text.Replace(@"</​th>",​ ";"​);​
 + text <- text.Replace(@"</​td>",​ ";"​);​
 + text <- text.Replace(@"</​tr>",​ Environment.NewLine);​
 + text <- text.Replace(',',​ '​|'​);​
 + text <- text.Replace('​.',​ ','​);​
 + text <- removeAllTags text
 + text
 +
 +let CSVresult = getCSVfromHTML (getDataFromHTML getWebpageHTML)
 +CSVresult.Dump()
 +</​code>​
 +</​hidden>​
 +
 +> An F# **type provider** is a component that provides types, properties, and methods for use in your program. ​
 +> The key to information-rich programming is to eliminate barriers to working with diverse information sources. One significant barrier to including a source of information into a program is the need to represent that information as types, properties, and methods for use in a programming language environment. ​
 +> Writing these types manually is very time-consuming and difficult to maintain. A common alternative is to use a code generator which adds files to your project; however, the conventional types of code generation do not integrate well into exploratory modes of programming supported by F# because the generated code must be replaced each time a service reference is adjusted.
 ===== References ===== ===== References =====
   * http://​fsharp.org/​   * http://​fsharp.org/​
projects/fsharp_workshop.txt ยท Last modified: 2015/05/02 12:45 by mkucia