Wednesday, August 15, 2012

C# CSV Mapping and WebClient Upload

In this tutorial, I'm going to give an example of creating a comma delimited csv formatted string from an object, and an example of uploading that object using WebClient. This is actually a continuation of my first post about replicating a curl request in C# without any extra libraries.

I'm using an object called MyObject.  This is a made up object for the example.  The fields in your classes must match those of the fields in your table in order for this code to work properly.

The line csvData = csvData.Substring(0, csvData.Length - 1); is used to cut off the last comma in each line of the csv output.
   
     public string MapCSVToTable(MyObject obj)

        {

            string csvData = "";


            // Loop through the object field names to use as the CSV header line.  
              //Descriptor allows us to get advanced information about an object such as field names and values.
            foreach (PropertyDescriptor descriptor in TypeDescriptor.GetProperties(obj.FirstOrDefault()))

            {
                
                csvData += descriptor.Name + ",";

            }

            csvData = csvData.Substring(0, csvData.Length - 1);


            //  Create a new line break.  
            csvData += Environment.NewLine;


             //  Loop through the object and get the values.
            foreach (var a in obj)

            {

                foreach (PropertyDescriptor descriptor in TypeDescriptor.GetProperties(a))

                {

                 

                    if (descriptor.GetValue(a) != null)

                    {

                        csvData += descriptor.GetValue(a) + ",";

                    }

                    else

                    {

                        csvData += ",";

                    }

                }

                csvData = csvData.Substring(0, csvData.Length - 1);
                
                // Create a new line break;
                csvData += Environment.NewLine;

            }

            return csvData;

        }

The UpdateTable function takes 3 parameters. The tableName specifies which table you would to affect, mappedData is the CSV string created in our first function, and httpMethod allows for you to specify "PUT" or "POST". "PUT" will update the record and "POST" will create a new record.
       public void UpdateTable(string tableName, string mappedData, string httpMethod)
        {
            // Create a new WebClient.
            WebClient client = new WebClient();
            // Encode the mapped CSV data into bytes.
            byte[] bytes = Encoding.UTF8.GetBytes(mappedData);
            
            string UserName = "username";
            string Password = "password";

            string authInfo = UserName + ":" + Password;

            client.Headers["Authorization"] = "Basic " + Convert.ToBase64String(Encoding.ASCII.GetBytes(authInfo));
            // Call UploadDate using your url, table name, method, and bytes created from your mapped data.
            byte[] responseBytes = client.UploadData("https://something.something/function/" + tableName, httpMethod, bytes);

            // The result is for debugging purposes.  It will show detailed error or success messages.
            string result = Encoding.UTF8.GetString(responseBytes);
        }
Usage example:


MyObject obj = new My Object();
obj.name = "Bruce Wayne";
obj.occupation = "Philanthropist Billionaire";
obj.emergencyContact = "Alfred Pennyworth";
obj.phone = "bat symbol";
obj.status = "MIA";

string mappedData = MapCSVToTable(obj);

UpdateTable("people", mappedData, "PUT");

CSV Example Output:


name, occupation, emergencyContact, phone, status
"Bruce Wayne", "Philanthropist Billionaire", "Alfred Pennyworth", "bat symbol", "MIA"

Monday, June 11, 2012

Basic Javascript News Rotator

This tutorial contains the skeleton of a plugin library, and an example of a basic news rotator, which seems to be all the rage these days.

The end result of the tutorial: Live Example

The first item MyObjects acts as a namespace for custom objects that you would like in your custom plugin library. The news rotator requires two variables, image and description. The image contains the location of the image that you would like to appear in the background of the rotator. The description contains the html or text that you would like to appear with the image.
var MyObjects = {};

MyObjects.news = function(){
 this.image = "";
 this.description = "";
};


MyLib will be the library name, which will contain our Widget library. Within the widget library, we will create a public method to return a newsRotator function, which takes information from a collection of unordered lists and creates the news rotator styling.
var MyLib = {};
MyLib.Widgets = (function() {
    // Public methods.
    return {
        newsRotator: function(autoPlay, ms) {
            // This adds the proper markup to create the rotator in the rotatorFrame placeholder.
            $(".rotatorFrame").html('<div id="rotatorPlaceHolder" class="rotatorPlaceHolder"><div class="rotatorHeader"><p></p></div></div><span class="rotatorLinks"></span>');
            // Grab all elements belonging to the class newsRotator.
            var list = $(".newsRotator");
            var newsCollection = [];

            //  Assign 8 seconds to the variable ms if a timing is not supplied.
            if (ms == null) {
                ms = 8000;
            }

            // Iterate through the list of unordered lists and create our news objects.
            $(list).each(function(index) {
                var news = new MyObjects.news();
                $(this).children("li").each(function(i) {
                    if (i === 0) news.image = $(this).html();
                    if (i === 1) news.description = $(this).html();
                });
                // Add the news object to an array, which houses a collection of news objects.
                newsCollection[index] = news;
            });

            var ncLength = newsCollection.length;
            var t = 0;

            // This sets the default information in the rotator.
            $(".rotatorPlaceHolder").css("background-image", "url(" + newsCollection[t].image + ")");
            $(".rotatorHeader p").html(newsCollection[t].description);

            // This table contains the navigation buttons for the rotator.
            $(".rotatorPlaceHolder").next("span").append("<table><tr>");

            // Cycle through the collection array, and assign a function to change the data to each created link.
            while (t < ncLength) {
                $(".rotatorPlaceHolder").next("span").append("<td><div id='imageLink" + t + "' class='newsClicked'></div></td>");
                //  The bind function uses an example of currying to maintain the state of the newsCollection array and t variable.
                $("#imageLink" + t).bind("click", (function(newsCollection, t) {
                    return function() {
                        $(".rotatorPlaceHolder").css("background-image", "url(" + newsCollection[t].image + ")").fadeOut(10).fadeIn(1000);
                        $(".rotatorHeader p").html(newsCollection[t].description);
                        if (autoPlay) {
                            $(".newsClicked").css("background-color", "rgb(231, 236, 245)");
                            $(this).css("background-color", "#00573C");
                        }
                        return false;
                    };
                })(newsCollection, t));
                t++;
            }

            var autoPlayInstance;
            var namespace = this;

            // Determine the state of autoPlay to supply the correct information to the autoPlay toggle button.
            if (autoPlay) {
                timer(0, ncLength);
                $(".rotatorPlaceHolder").next("span").append("<td>&nbsp&nbsp<a href='#' id='autoPlayBtn'>Autoplay On</a></td>");
                $("#autoPlayBtn").bind("click", function() {
                    clearTimeout(autoPlayInstance);
                    $(".rotatorPlaceHolder").next("span").html("");
                    namespace.newsRotator(false);
                    return false;
                });
            }
            else {
                $(".rotatorPlaceHolder").next("span").append("<td>&nbsp&nbsp<a href='#' id='autoPlayBtn'>Autoplay Off</a></td>");
                $("#autoPlayBtn").bind("click", function() {
                    clearTimeout(autoPlayInstance);
                    $(".rotatorPlaceHolder").next("span").html("");
                    namespace.newsRotator(true);
                    return false;
                });
            }

            // End the table of buttons.
            $(".rotatorPlaceHolder").next("span").append("</tr></table>");

            // Create a function to keep track of timing if autoplay is true.


            function timer(i, length) {
                if (autoPlay) {
                    $("#imageLink" + i).trigger("click");
                    i++;
                    if (i > ncLength) {
                        i = 0;
                    }
                    autoPlayInstance = setTimeout(function() {
                        timer(i, length);
                    }, ms);
                }
            }
        }
    };
})();​


The html for the news rotator consists of placeholder div with a collection of unordered lists. The first list item contains the path to the image, while the second list item contains the html that you would like to appear in the text section of the rotator. The html could be further simplified, but I stuck with this format for the sake of this example.
  • Images/Desert.jpg
  • Desert Header

    click this some test text some test some test text some test some test text some test some test text some test some test text some test some test text some test some test text some test some test text some test some test text some test some test text some test some test text some test some test text some test some test text some testsome test text some test text some test text some test text some test text some test text some test text some test text some test text some test text some test text
  • Images/Koala.jpg
  • Koala

    description1
  • Images/Chrysanthemum.jpg
  • Test

    description2
  • Images/Jellyfish.jpg
  • Jelly Fish

    click this some test text some test text some test text some test text some test text some test text some test text some test text some test text some test text some test text
  • Images/Penguins.jpg
  • Penguins

    description1
  • Images/Lighthouse.jpg
  • Lighthouse

    description2


This is a stylesheet to include with the javascript code above. Feel free to change the size, colors and other attributes. This css works in all current versions of Firefox, IE, Chrome, Safari and Opera.
.newsRotator
{
 display: none;
}
.rotatorPlaceHolder
{
 width: 760px;
 height: 500px;
 background-color: black;
 z-index: -1;
 border-radius: 7px 7px 7px 7px;   
}
.rotatorHeader
{
 opacity: 0.5;
 filter: alpha(opacity=50);
 position: relative;
 color: white;
 /*width: 760px;*/
 width: 100%;
 height: 150px;
 overflow: auto;
 background-color: black;
    top: 70%;
}
.rotatorHeader:hover
{
 opacity: 0.75;
 filter: alpha(opacity=75);
}
.rotatorHeader h3
{
 padding: 10px 0px 10px 10px;
 color: white;
}
.rotatorHeader p
{
 padding: 0px 10px 10px 10px;
 color: white;
}
.rotatorHeader a
{
 color: white;
}
.rotatorFrame
{
 background-color: white;
 padding: 25px 25px 25px 25px;
 height: 550px;
 width: 760px;
 border-radius: 20px 20px 20px 20px;
 border: 1px outset gray;
 box-shadow: 10px 10px 5px #888888;
}
.rotatorLinks 
{
 margin-right: auto;
 margin-left: auto;
 padding-bottom: 5px;
 width: 760px;
}
.rotatorLinks div
{
 position: relative;
 margin: 5px 5px 5px 5px;
 background-color: rgb(231, 236, 245);
 width: 25px;
 height: 25px;
 border-radius: 20px 20px 20px 20px;
}
.rotatorLinks div:hover
{
 background-color:  #00573C;
}
.rotatorLinks div:active
{
 background-color:  #00573C;
}
#autoPlayBtn
{
   color: #006E51;
   font-size: 11px;
   font-family: Helvetica, Arial, Sans-Serif;
   text-decoration: none;
   font-weight:bold;  
}
#autoPlayBtn:hover 
{
   color: #00573C;
   text-decoration: underline;
}
#autoPlayBtn:active 
{
   color: #00573C;
}

body
{
    font-family: Helvetica, Arial, Sans-Serif;
}


This usage of newsRotator will load the plugin with autoPlay and the default 8000 ms between content. In order to use this plugin, you need to have JQuery included in your project. I tested with version 1.7.1.





Friday, June 8, 2012

WebRequest Imitating Curl in C#

This is a quick tutorial on creating a WebRequest to imitate a Curl GET in C#. Curl is a fast and simple way to GET and post CSV data. I had previously looked at using IronPython, a python library for .net, in my project to directly call python, but the library doesn't support pycurl yet. Since Curl and Python aren't installed on Windows systems by default, I decided to use a WebRequest because it makes the code more portable, and doesn't require any extra installation steps on the server. You can still use this code on Linux and Mac systems, but that would require using Monocode.

After the WebRequest, there's also a short bit on usage where I will show you how to use the JavaScriptSerializer to deserialize the JSON string into a predefined object. This will enable you to use Linq on the object, and open up a wealth of uses of the data within your app. Limiting Curl requests for data in favor of building a data model will result in faster processing of the data.

getRecordsByTable returns a built JSON string to return the data from your supplied table name.
public class CurlTest
{
   public string getRecordsByTable(string tableName)
   {
               // build the request url with the tableName appended to the end.
               WebRequest request = WebRequest.Create("https://hostsite/" + tableName);

               // use the GET method.
               request.Method = "GET";

        // get this information from settings in your web config.
               string userName = "someone@somesite.something";
               string password = "password";

               string credentials = userName + ":" + password;
               request.Headers["Authorization"] = "Basic " + Convert.ToBase64String(Encoding.ASCII.GetBytes(credentials));

        //  this is a hack to test ssl without a certificate.
               ServicePointManager.ServerCertificateValidationCallback = new System.Net.Security.RemoteCertificateValidationCallback(AcceptAllCertifications);
   
   // create a web response
               WebResponse response = request.GetResponse();
   
   // create a data stream.
               Stream dataStream = response.GetResponseStream();
   
               // create a stream reader.
               StreamReader reader = new StreamReader(dataStream);
   
               // read the content into a string
               string serverResponse = reader.ReadToEnd();
   
   // map the CSV data to a JSON string.
               var mappedData = mapTableToJSON(serverResponse);
   
               // clean up.
              reader.Close();
              dataStream.Close();
              response.Close();

              return mappedData;
   }
}


The AcceptAllCertifications callback returns true to acknowledge that the host SSL certificate is valid. I wouldn't recommend this approach on production, but it's a great way to test a secure request over https. If you want to do a request over an unsecure site, just change https to http in your WebRequest url, and leave out the authentication parts of the code.
        public bool AcceptAllCertifications(object sender, System.Security.Cryptography.X509Certificates.X509Certificate certification, System.Security.Cryptography.X509Certificates.X509Chain chain, System.Net.Security.SslPolicyErrors sslPolicyErrors)
        {
            return true;
        }


The mapTableToJSON method takes the CSV string data, and formats that data into a JSON string.
public static string mapTableToJSON(string value)
        {
            // get lines out of the CSV data.
            if (value == null) return null;
            string[] lines = value.Split('\n');

            // get headers out of the CSV data.
            string[] headers = lines.First().Split(',');

            // build the JSON string.
            StringBuilder sb = new StringBuilder();
   // open the JSON formatting.
            sb.AppendLine("[");
   // iterate through the lines for fields.
            for (int i = 1; i < lines.Length; i++)
            {
    // regex selects comma delimiters that are not in quotes in order to avoid syntax issues because some fields may include a comma in the value.
                string[] fields = Regex.Split(lines[i], @",(?=(?:[^""]*""[^""]*"")*(?![^""]*""))");
                int a = 0;
                foreach (var f in fields)
                {
     // insert empty quotes as a JSON value for a header if the CSV value is empty.
                    if (String.IsNullOrEmpty(f))
                    {
                        fields[a] = "\"\"";
                    }
                    a++;
                }
    // format the headers and fields and output to an array.
                var elements = headers.Zip(fields, (header, field) => string.Format("{0}: {1}", header, field)).ToArray();
                string json = "{" + string.Format("{0}", string.Join(",", elements)) + "}";
    // apply commas after each json object except the last one to avoid syntax errors.
                if (i < lines.Length - 1)
                {
                    json += ",";
                }
    // append the JSON to the string builder.
                sb.AppendLine(json);
            }
   // close the JSON formatting.
            sb.AppendLine("]");

            return sb.ToString();
        }


Here is an example of usage, and an example of mapping the JSON string to our predefined Person object. The JSON header values must literally match your object variables.
public class Person
{
   public int id { get; set; }
   public string name { get; set; }
   public string comments { get; set; }
}

JSON string example: 
[
        {"id": "1", "name": "Darth Vader", "comments": "I find your lack of faith disturbing."},
        {"id": "2", "name": "Obi Wan Kenobi", "comments": "Sir Alec Guinness is awesome."},
        {"id": "3", "name": "Jar Jar Binks", "comments": "We can always delete him here."}
]

getAllPersonData is a method to show you how to map your JSON string to our predefined Person object.
public List<Person> getAllPersonData()
{
   // create an instance of the CurlTest class.
   CurlTest curlTest = new CurlTest();

   // use the getRecordsByTable method to get mapped JSON data.
   string jsonText = curlTest.getRecordsByTable("person");

   JavaScriptSerializer serializer = new JavaScriptSerializer();
   // if you have an issue with MaxJsonLength, make the length a higher number than the default.

   // deserialize the JSON object into a predefined object.
   List<Person> person = serializer.Deserialize<Person>(jsonText);

   //  :-)
   person.RemoveAll(a => a.name == "Jar Jar Binks");

   return person;
}

I will eventually include using parameters in the GET request, and show an example of posting CSV data.