Advertisements
jump to navigation

Dynamically Create Test Data with NBuilder, JSON and .Net October 24, 2010

Posted by ActiveEngine Sensei in .Net, ActiveEngine, Ajax, ASP.Net, C#, Fluent, LINQ, Open Source, Problem Solving.
Tags: , , , , ,
5 comments

Part 1 of a 3 part series.  For the latest DataBuilder capabilities, read this post or download the new source code from here.

Building test data should be as easy:

var agentList = Builder<SalesAgent>.CreateListOfSize(5)
                           .WhereTheFirst(1)
                                  .Have(x => x.FirstName = "James")
                                  .And(x => x.LastName = "Kirk")
                            .AndTheNext(1)
                                  .Have(x => x.FirstName = "Bruce")
                                  .And(x => x.LastName = "Campbell")
                            .Build()
                            .ToList();

Wouldn’t be nice if all the properties of your objects were automatically populated:

Product:
       Id              : 1
       Title           : "Title1"
       Description     : "Description1"
       QuantityInStock : 1

NBuilder by provides you with a great fluent interface to accomplish this with ease.  You can even achieve scenarios where you can create hierarchies of data, set property values on a range objects in a list, and even create a specified range of values that you can use populate other objects.  Peruse through the samples and you will see, NBuilder quite capably maps values  the public properties of your objects.  A real time saver.

Sensei is going to kick it up a notch and provide you with a means to create test data with out having to recompile your projects.  This is ideal for when you want to create UI prototypes.  DataBulider uses CS-Script and NBuilder to create a web based data generation tool that can read assemblies and will allow you to script a process that will generate test data in the form of JSON.

This adventure is split into two parts.  First a quick demo, then instructions on how to configure DataBuilder for you environment.  A deeper discussion of CS-Script and embedded scripting in .Net will be part of the sequel to this action/adventure, as we all know the second movie in the series is always the best!.

Operating DataBuilder

In short you have three things to do:

  • Identify the assemblies that contains the objects you want to generate test data for.  The path to the files can be anywhere on your system.  For convenience there is an folder called Assembly that you can copy the files to.  Multiple assemblies from different locations can be imported.
  • Create the import statements.
  • Create the code snippet with the NBuilder statements that will generate your data.

Here’s a screen shot of DataBuilder with each section that corresponds with the three goals stated above.

And here is an example that we’ll be working with.

var agents = Builder<SalesAgent>.CreateListOfSize(5)
                    .WhereTheFirst(1)
                         .Have(x => x.FirstName = "James")
                         .And(x => x.LastName ="Kirk")
                    .AndTheNext(1)
                          .Have(x => x.FirstName = "Bruce")
                          .And(x => x.LastName = "Campbell"})
                    .Build()
                    .ToList();

parameters["JsonDataSet"] = JsonConvert.SerializeObject(agents);

Note that after the end of the code that creates the objects, you need to include a statement

parameters["JsonDataSet"] = JsonConvert.SerializeObject(List);

Without that statement you will not get your data serialized.  If you’ve entered the data as shown, hit the Build button and the resulting JSON is placed in the output box.  That’s it.  Looking through the output you’ll note that the first two sales dudes are James Kirk and Bruce Campbell, while the remaining records are completed by NBuilder.

[{"FirstName":"James","LastName":"Kirk","Salary":1.0,"RegionId":1,"RegionName":"RegionName1","StartDate":"\/Date(1287892800000-0400)\/"},{"FirstName":"Bruce","LastName":"Campbell","Salary":2.0,"RegionId":2,"RegionName":"RegionName2","StartDate":"\/Date(1287979200000-0400)\/"},{"FirstName":"FirstName3","LastName":"LastName3","Salary":3.0,"RegionId":3,"RegionName":"RegionName3","StartDate":"\/Date(1288065600000-0400)\/"},{"FirstName":"FirstName4","LastName":"LastName4","Salary":4.0,"RegionId":4,"RegionName":"RegionName4","StartDate":"\/Date(1288152000000-0400)\/"},{"FirstName":"FirstName5","LastName":"LastName5","Salary":5.0,"RegionId":5,"RegionName":"RegionName5","StartDate":"\/Date(1288238400000-0400)\/"}]

You also can load a script and execute it as well.  That’s done on the “Script Loader” tab.  The location of the scripts is set in the WebConfig and the key name is ScriptPath.  Here’s the screen shot:

Anatonomy of DataBuilder Script

Here’s the complete C# script file that builds your data.  It’s just a class:

//CSScript directives - DO NOT REMOVE THE css_ref SECTION!!!
//css_ref System.Core;
//css_ref System.Data.ComponentModel;
//css_ref System.Data.DataSetExtensions;
//css_ref System.Xml.Linq;

using System;
using System.Collections.Generic;
using System.Text;
using System.IO;
using DataGenerator.Core;
using DataGenerator.ObjectTypes;
using DataGenerator.ScriptHost;
using System.Linq.Expressions;
using System.Linq;
using Newtonsoft.Json;
using FizzWare.NBuilder;
//  Add a reference to your assemblies as well!!
using UserDeploymentDomain;

public class CreateTestFile : IScriptRunner
{
    public void  RunScript(Dictionary parameters)
    {
        var agents = Builder.CreateListOfSize(5)
                    .WhereTheFirst(1)
                         .Have(x => x.FirstName = "James")
                         .And(x => x.LastName = "Kirk")
                    .AndTheNext(1)
                          .Have(x => x.FirstName = "Bruce")
                          .And(x => x.LastName = "Campbell")
                    .Build()
                    .ToList();

        parameters["JsonDataSet"] = JsonConvert.SerializeObject(agents);
    }

    public void  RunScript()
    {
 	    throw new NotImplementedException();
    }
}

The very top section “CSScript Directives” is required by CS-Script.  These are directives that instruct the CS-Script engine to include assemblies when it compiles the script.  The imports section is pretty straight forward.

You’ll note that the script inherits from an interface.  This is a convention used by CS-Script to allow the host and script to share their respective assemblies.  Sensei will discuss that in next post.  The RunScript method accepts a Dictionary that contains the parameters.  This will house the JsonDataSet that is expected for the screen to display the output of your data.

Advanced NBuilder Experiments
The beauty of NBuilder is that you can create test data that goes beyond “FirstName1”, and allows you to quickly create data that matches what the business users are used to seeing. If you think about it you should be able to generate test data that will exercise any rules that you have in the business domain, such as “Add 5% tax when shipping to New York”. With the scripting capability of DataBuilder you can create suites test data that can evolve as you test your system. You could also use the JsonDataSet to create mocks of your objects as well, maybe use them for prototyping your front end.

We’ll do a quick sample. Our scenario is to create assign real regions to sales agents. Furthermore, we want to only chose a range of regions and assign them at random.

First we build the Regions:

var regions= Builder<Region>.CreateListOfSize(4)
	.WhereTheFirst(1)
		.Have(x => x.State = "Texas")
	.AndTheNext(1)
		.Have(x => x.State = "California")
	.AndTheNext(1)
		.Have(x => x.State = "Ohio")
	.AndTheNext(1)
		.Have(x => x.State = "New York")
	.Build();

Now we’ll create a SalesAgents and using the Pick method from NBuilder we’ll randomly assign a region to the sales agents:

var agents = Builder<SalesAgent>.CreateListOfSize(5)
                    .WhereAll()
                           .HaveDoneToThem(x => x.RegionName = Pick.RandomItemFrom(regions).State)
                    .WhereTheFirst(1)
                         .Have(x => x.FirstName = "James")
                         .And(x => x.LastName = "Kirk")
                    .AndTheNext(1)
                          .Have(x => x.FirstName = "Bruce")
                          .And(x => x.LastName = "Campbell")
                    .Build()
                    .ToList();

The result set now has the range of states distributed to the Sales Agents. Looks like James Kirk needs to cover Texas. You may need to view the source to see the output.

[{"FirstName":"James","LastName":"Kirk","Salary":1.0,"RegionId":1,"RegionName":"Texas","StartDate":"\/Date(1287892800000-0400)\/"},{"FirstName":"Bruce","LastName":"Campbell","Salary":2.0,"RegionId":2,"RegionName":"Texas","StartDate":"\/Date(1287979200000-0400)\/"},{"FirstName":"FirstName3","LastName":"LastName3","Salary":3.0,"RegionId":3,"RegionName":"California","StartDate":"\/Date(1288065600000-0400)\/"},{"FirstName":"FirstName4","LastName":"LastName4","Salary":4.0,"RegionId":4,"RegionName":"California","StartDate":"\/Date(1288152000000-0400)\/"},{"FirstName":"FirstName5","LastName":"LastName5","Salary":5.0,"RegionId":5,"RegionName":"Ohio","StartDate":"\/Date(1288238400000-0400)\/"}]

Configure DataBuilder For Your Environment
Given that DataBuilder is loading assemblies you will want to run it on either your dev environment or on a test server where your co workers won’t mind if you need to take IIS up and down. Also, you’ll want to work with a copy of your assemblies in case you need to make a quick change. There are times when IIS will not release a file and if you need to make changes to the assemblies themselves it’s more convenient to copy them after you’ve re-compiled.

There are two settings you need to change in the WebConfig to match your environment.

ScriptPath – Point this to the share where you want to save any scripts. DataBuilder will scour the directory and list anything you place in there.

FizzWarePath – This needs to point to the location of the NBuilder dll. Most likely this will be the bin folder of the DataBuilder website. In the follow up post Sensei will explain what this does.

Wrapping Up For Now

We covered a lot on the whirlwind tour of DataBuilder.  There’s a lot more that is of interest, particularly with respects to the embedded scripting aspects provided by CS-Script.  For now, have fun playing building you data sets.  In the next installment we’ll cover the scripting aspect in more detail  For now, download and experiment.  Here’s the source for DataBuilder with unit tests.

Advertisements

Deserializing to Persistent AnonymousTypes with JSON.Net October 9, 2010

Posted by ActiveEngine Sensei in .Net, .Net Development, ActiveEngine, C#, Problem Solving.
Tags: , , ,
1 comment so far

A few weeks back Sensei unleashed a crazy idea regarding a class AnonymousType that could persist values from an anonymous object.  The AnonymousType, created by Hugo Benocci models an individual object.  In a sense this is a hyper-charged Dictionary of properties that represent an object.  It’s meta data.  This is similar to a concept called the Adaptive Object Model, the theory that you create mechanisms to describe what your objects should do.   Instead of having a class for SalesAgent or Car you have classes that represent the classes, attributes, relationships and behavior in your domain.  In other words, you create a meta data modeler and feed it the criteria that would represent SalesAgent, Car, etc.

Having a “sound-of-one-hand-clapping” moment, Sensei realized that while “Persistent AnonymousTypes” was in the title of the post, no mechanism for for serializing the AnonymousType was included!!  “What the …”.  Jeeezz!  “Hell, that should be easy”, Sensei says.  Grab JSON.Net and with elbow grease make it work, right?  Anybody?

One thing that should be immediately clear is that all the meta data is locked up in the AnonymousType object, so you can’t just write:

string json = JsonConvert.SerializeObject(anonymousType);

Instead we need a way represent all the properties of our AnonymousType and preserve each property’s name, it’s type, and the underlying value.  Something like:

public class NameTypeValue
{
  public string Name { get; set; }
  public Type Type{get; set;}
  public object Value { get; set; }
}

And wouldn’t it be nice if we could take a serialized stream of an actual object and convert that into an AnonymousType?  Thinking further ahead, it would rather easy to pass around a list of NameTypeValues as you could easily send and receive this object from a web client or other front end, building yourself a modelling or code generation tool.

Serializing the object depicted above is pretty trivial.  Using a Func<Dictionary<string,object>, string,  string> we can serialize any way wish as with two tiny methods:

public string ToJSON(Func, string, string> function, string jsonObjectName)
{
    return function(_Values, jsonObjectName);
}
///  Method to serialize.  You can come up with your own!!
public string SerializeWithJObject(Dictionary values, string name)
{
  var jsonObject = new JObject();

  foreach (KeyValuePair property in values)
  {
    jsonObject.Add(new JProperty(property.Key, property.Value));
  }

  return jsonObject.ToString();
}

If there is another mechanism for serialization that you wish to use you are free to come up with your own.  For illustration here is the JSON output of an AnonymousType for a sales agent, and followed by the JSON for an actual Agent object:

Agent JSON ==>{“Name”:”Sales Guy Rudy”,”Department”:45}

AnonymousType JSON ==>{  “Name”: “Sales Guy Rudy”,  “Department”: 45}

 

Now that we can simply serialize our AnonymousType with the output matching that of an actual object,  we just need a way to interpret a JSON stream and build an AnonymousType.  Along with discussion, Sensei will talk about the second “sound-of-one-hand-clapping” moment he had when working with JSON.Net.  As you may have already surmised, you need to describe the Type of property in order deserialization to happen.  Sensei didn’t and took a trip to the valley for frustration.

Ok.  We have stream of JSON with the Name, Value and Type of each property for an object.  AnonymousType has a Set method to set a new property:

        /// <summary>
        /// Sets the value of a property on an anonymous type
        /// </summary>
        /// <remarks>Anonymous types are read-only - this saves a value to another location</remarks>
        public void Set(string property, object value) {
            this.Set<object>(property, value);
        }

        /// <summary>
        /// Sets the value of a property on an anonymous type
        /// </summary>
        /// <remarks>Anonymous types are read-only - this saves a value to another location</remarks>
        public void Set<T>(string property, T value) {

            //check for the value
            if (!this.Has(property)) {
                this._Values.Add(property, value);

            }
            else {

                //try and return the value
                try {
                    this._Values[property] = value;
                }
                catch (Exception ex) {
                    throw new Exception(
                        string.Format(
                            AnonymousType.EXCEPTION_COULD_NOT_ACCESS_PROPERTY,
                            property,
                            (value == null ? "null" : value.GetType().Name),
                            ex.Message
                            ),
                            ex);
                }
            }

        }

It’s pretty straight forward to accept a NameTypeValue object and perform:

public void AddProperty(string objectName, NameTypeValue nameTypeValue)
{
 //  Object doesn't exist?  Add.
 if (objects.ContainsKey(objectName) == false)
 {
 objects.Add(objectName, new List());
 }

 var properties = objects[objectName];

 //  All properties are unique
 var existingProperty = properties.Where(x => x.Name == nameTypeValue.Name)
 .SingleOrDefault();

 if(existingProperty == null)
 {
 properties.Add(nameTypeValue);
 }
}

and taking this a step further, a List<NameTypeValue> can supply all properties for an object:

properties.ForEach(x => { anonymousType.Set(x.Name, x.Value); });

Accepting a JSON stream of a List<NameTypeValue> should be easy-cheesey mac-n-peasey.  The first version of this looked like the following:

public AnonymousType DeserializeFromJSONProperties(string objectName, string json)
{
  Enforce.ArgumentNotNull(objectName, "AnonFactory.Deserialize - objectName can not be null");
  Enforce.ArgumentNotNull(json, "AnonFactory.Deserialize - json can not be null");

  List propertyList = JsonConvert.DeserializeObject
>(json);

  //  Add properties.  Make sure int is not deserialized to a long since JSON.Net
  //  makes best guess
  propertyList.ForEach(x => AddProperty(objectName, x));

  return CreateAnonymousType(objectName);
}

But one-moooorrree-thing!  Sensei discovered that JSON.Net, when presented with an integer like 5, will deserialize to the largest possible type when not presented with a target.  In other words, when you have this JSON:

{“Department” : 45}

and deserialize to an object, it must accommodate the largest possible type in order to avoid truncating the data.  That means an int is deserialized as Int64!!  The first round of testing was quite aggravating as AnonymousType would accept the property into it’s schema, but when you went to fetch that value later on you would get an exception  In other words, when you did this:

//  Found in JSONTests.MakeItFail()
var anonFactory = new AnonFactory();
var darrellDept = new NameTypeValue();
darrellDept.Name = "Department";
darrellDept.Value = 45;

var darrellName = new NameTypeValue();
darrellName.Name = "Name";
darrellName.Value = "Darrell";

var propertyList = new List();
propertyList.Add(darrellDept);
propertyList.Add(darrellName);

//  Create JSON stream of properties
string darrellPropertyJSON = JsonConvert.SerializeObject(propertyList);

//  Try to deserialize and create an AnonymousType object
var otherDarrell = anonFactory.DeserializeFromJSONProperties("Agent", darrellPropertyJSON);
Assert.AreEqual(otherDarrell.Get("Department"), 45);

you got an InvalidCastException.

Luckily you have the Type so you can perform a conversion as you deserialize the property and add it to AnonymousType’s Dictionary<string, object>.  Here’s the new version:

propertyList.ForEach(x => AddProperty(objectName, ConvertTypeFromDefinition(x)));

private NameTypeValue ConvertTypeFromDefinition(NameTypeValue nameTypeValue)
{
  if (nameTypeValue.Type != nameTypeValue.Value.GetType())
  {
    nameTypeValue.Value = Convert.ChangeType(nameTypeValue.Value, nameTypeValue.Type);
  }

  return nameTypeValue;
}

When you look at the new version of the AnonymoustType project you’ll note that serializing is handled by the AnonymousType itself, while a factory class is used for building the an AnonymousType from the NameTypeValue’s and for deserializing JSON as well.  Sensei struggled a bit with this, as on the one hand if AnonymousType was responsible for serializing itself should it also be able to deserialize a stream?  On the other hand, a factory seemed logical since you could have a registry of AnonymousType objects, thereby centralizing the creation and management of AnonymousTypes.  Don’t like it – create your own and share!  Regardless, looks like we can fit through the mini-Stargate now.  Here’s version 2.

%d bloggers like this: