Differences

This shows you the differences between two versions of the page.

--- projects:fsharp_workshop [2015/04/18 17:32] – mkucia
+++ projects:fsharp_workshop [2015/05/02 12:45] (current) – [hello(x)] mkucia
@@ Line 92: / Line 92: @@
 Note that F# uses indentation to indicate block structure. Find out more [[http://fsharpforfunandprofit.com/posts/fsharp-syntax/|here]].
+===== Getting data =====
+The following example loads data from webpage and stores it in CSV file.
+<hidden>
+<code fsarp>
+open System.Net
+open System
+open System.IO
+let myURL = @"..."
+let fetchUrl callback url =
+    let req    = WebRequest.Create(Uri(url))
+    use resp   = req.GetResponse()
+    use stream = resp.GetResponseStream()
+    use reader = new IO.StreamReader(stream)
+    callback reader url
+let myCallback (reader:IO.StreamReader) url =
+    let htmlSource = reader.ReadToEnd()
+    htmlSource
+let getWebpageHTML = fetchUrl myCallback myURL
+type FirstOrLast =
+	| GetFirst
+	| GetLast
+let getDataFromHTML (rawHTML:String) =
+	let split (split_point:String) (firstOrLast:FirstOrLast) (text:String) =
+		let array = text.Split([|split_point|], 2, StringSplitOptions.RemoveEmptyEntries)
+		match firstOrLast with
+		| GetFirst -> array.First()
+		| GetLast  -> array.Last()
+	rawHTML
+	|> split @"<table id=""t2"">" GetLast
+	|> split @"</table>"          GetFirst
+let removeAllTags text =
+    Regex.Replace(text, "<.*?>", String.Empty);
+let getCSVfromHTML rawHTML =
+	let mutable (text:String) = String.Empty
+	text <- rawHTML
+	text <- text.Replace(@"</th>", ";");
+	text <- text.Replace(@"</td>", ";");
+	text <- text.Replace(@"</tr>", Environment.NewLine);
+	text <- text.Replace(',', '|');
+	text <- text.Replace('.', ',');
+	text <- removeAllTags text
+	text
+let CSVresult = getCSVfromHTML (getDataFromHTML getWebpageHTML)
+CSVresult.Dump()
+</code>
+</hidden>
+> An F# **type provider** is a component that provides types, properties, and methods for use in your program.
+> The key to information-rich programming is to eliminate barriers to working with diverse information sources. One significant barrier to including a source of information into a program is the need to represent that information as types, properties, and methods for use in a programming language environment.
+> Writing these types manually is very time-consuming and difficult to maintain. A common alternative is to use a code generator which adds files to your project; however, the conventional types of code generation do not integrate well into exploratory modes of programming supported by F# because the generated code must be replaced each time a service reference is adjusted.
 ===== References =====
   * http://fsharp.org/