Advertisements
jump to navigation

Develop A Theme For Your Project – Spock Can Help April 28, 2013

Posted by ActiveEngine Sensei in ActiveEngine, Mythology, New Techniques, Personal Development, Problem Solving, software economics.
Tags: , , , , , , ,
add a comment

Evil_SpockMany times Sensei has said you have to have a theme song for your projects. You may have certainly noticed that Sensei is old school, prog-rock and somewhat metal oriented. Spock’s Beard is a recent discovery and the group has direct roots with Transatlantic.

This latest album is a great source of inspiration, so if you have a ten minute walk ahead of, fire it up and it will get your head straight for serious productivity, creativity, or pure coding marathons.

Advertisements

The Speed of Thought, Part Duh!!! (Or Driving Stick Shift with Javascript) April 18, 2013

Posted by ActiveEngine Sensei in ActiveEngine, Coaching, Humor, jQuery, KnockoutJS, Mythology, Problem Solving, Scripting.
Tags: , , , , ,
add a comment

scream_baron_blood_01Sensei is a libertarian – so you can interpret that to mean few rules, respect others freedom, government and busy bodies “leave me the hell alone”, stay-outta-my-way-attitude-person is his motto.  That means the only way to drive is with the stick.  Automatic is for the soccer moms.  Javascript is like driving manual transmission – sometimes you grind the gears.

You’re hero just spent a half hour pouring over some Knockout.js code, wondering why his observableArray went MIA on each push.  Can you see why?

var Question = function (id, question, sortOrder) {
this.Id = id;
this.Question = ko.observable(question);
this.SortOrder = ko.observable(sortOrder);
};

It’s not Javascript’s fault, it’s Sensei’s fault. Now you can see why he writes in the third person, ’cause it’s easier to remove yourself from these type of dumb mistakes when you can treat your persona as separate person!! If you see the issue, leave a comment before I post the resolution.

The Speed of Thought April 17, 2013

Posted by ActiveEngine Sensei in ActiveEngine, Approvaflow, Business Processes, Coaching, Fluent, Mythology, Personal Development, Problem Solving.
Tags: , , , , , , , ,
add a comment

faradaypennytwinsOf late, Sensei needs to keep a clear head.  That has meant learning to segment ideas and really, really, really focus on streamlined features.  This is hard.  Not because Sensei has a plethora of great ideas.  That would be a nice problem to have.  Many times in software development you end up this guy:

This is the state where you have things you want to accomplish, yet even when you pair things down to the “essential”, other essential, critical factors must be taken into consideration before you create a mess.  This is life calling, and that string which suspends that giant sword that you noticed hovering over your head is about to snap.  There is a good chance that you need more discipline, better execution tactics, far better honed chops, you name the metaphor.  Sensei has been at this game for over 22 years, and still the speed that thought takes to become reality is way too slow.

With great sarcasm you can remind your self that some of the best work lays ahead, but the reality is that you still need to fight to be fluent, you have to claw your way to a Zen state of mind / no-mind.  So chose, the art of bushido or the art of BS.  Or maybe work smarter and enjoy life.

Before Sensei leaves you, ponder this:  does “being done” mean that you’ve dropped off a product but have to get on the phone in order to make changes, and maybe now that you are struggling why couldn’t you figure out to take time when it was more critical to be fluent with your productivity?

 

A Life Time In a Day April 5, 2013

Posted by ActiveEngine Sensei in ActiveEngine, Coaching, Mythology, Problem Solving.
Tags: ,
3 comments

If you are lucky, you can experience a life through the course of a day. Consider it the antithesis of ground hogs day. Instantaneous history, melancholy,!and certainty all at once. You are here, where do you go?

20130405-174801.jpg

Getting KO’ed with KnockoutJS August 31, 2012

Posted by ActiveEngine Sensei in ActiveEngine, Ajax, Approvaflow, ASP.Net, DataTables.Net, jQuery, New Techniques, Open Source.
Tags: , , , , ,
4 comments

ImageOn the quest to provide a rich user interface experience on his current project, Sensei has been experimenting with KnockoutJS by Steve Sanderson.  If you haven’t reviewed it’s capabilities yet  it would be well worth your while.  Not only has Steve put together a great series of tutorials, but he has been dog fooding it with Knockout.  The entire documentation and tutorial set is completed used Knockout.  Another fine source is Knockmeout.net by Ryan Niemeyer.  Ryan is extremely active on StackOverflow answering questions regarding Knockout, and also has a fine blog that offers very important insight on developing with this framework.

KnockoutJS is a great way to re-organize your client side code.  The goal of  this post is not to teach you KnocoutJS; rather, Sensei wants to point out other benefits – and a few pitfalls – to adopting its use.  In years past, it’s been difficult to avoid writing spaghetti code in Javascript.  Knockout forces you to adopt a new pattern of thought for organizing your UI implementation.  The result is a more maintainable code base.  In the past you may have written code similar to what Sensei use to write.  Take for example assigning a click event to a button or href in order to remove a record from a table:

<table>
  <thead></thead>
  <tbody>
    <tr>
      <td><a onclick="deleteRecord(1); return false;" href="#">Customer One</a></td>
      <td>1313 Galaxy Way</td>
    </tr>
    <tr>
      <td><a onclick="deleteRecord(2); return false;" href="#">Customer Two</a></td>
      <td>27 Mockingbird Lane</td>
    </tr>
</tbody>
</table>

<script type="text/javascript">
function deleteRecord(id){
  //  Do some delete activities ...
}
</script>

You might even went as far as to assign the onclick event like so:

$(document).ready(function(){
  $("tr a").on('click', function(){
    //  find the customer id and call the delete record
  });
});

The proposition offered by Knockout is much different.  Many others much more conversant in design patterns and development than Sensei can offer better technical reasons why you sound use Knockout.  Sensei likes the fact that it makes thinking about your code much simpler.  As in:

 
<td><a data-bind="click: deleteRecord($data)" href="#">Customer One</a></td>

Yep, you have code mixed in with your mark up, but so what.  You can hunt down what’s going on, switch to your external js file to review what deleteRecord is supposed to do.  It’s as simple as that.  Speaking of js files, Knockout forces you to have a more disciplined approach organizing your javascript.  Here is what the supporting javascript could look like:

var CustomerRecord = function(id, name){
  //  The items you want to appear in UI are wrapped with ko.observable
  this.id = ko.observable(id);
  this.name = ko.observable(name);
}

var ViewModel = function(){
var self = this;
  //  For our demo let's create two customer records.  Normally you'll get Json from the server
  self.customers = ko.observableArray([
    new CustomerRecord(1, "Vandelay Industries"),
    new CustomerRecord(2, "Wiley Acme Associates")
  ]);

  self.deleteRecord = function(data){
    //  Simply remove the item that matches data from the array
    self.customers.remove(data);
  }
}

var vm = new ViewModel();
ko.applyBindings(vm);

That’s it.  Include this file with your markup and that’s all you have to do.  The html will change too.   Knockout will allow you to produce our table by employing the following syntax:

<tbody data-bind=”foreach: customers”>
<tr>
<td><a href=”#” data-bind=”click:  deleteRecord($data)”><span data-bind=”text: id”></span></a></td>
<td><span data-bind=’text: name”></span></td>
<tr>
</tbody>

These Aren’t the Voids You’re Looking For

So we’re all touchy feely because we have organization to our Javascript and that’s a good thing.  Here’s some distressing news – while Knockout is a great framework, getting the hang of it can be really hard.  Part of the reason is Javascript itself.  Because it’s a scripting language, you end up with strange scenarios where you have a property that appear to have the same name but different values.  You see, one of the first rules of using Knockout is that observables ARE METHODS.  You have to access them with (), as in customer.name(), and not customer.name.  In other words, in order for you to assign values to an observable you must:


customer.name("Vandelay Industries");

//  Don't do this - you create another property!!

customer.name = "Vandelay Industries";

What? Actually, as you probably have surmised, you get .name() and .name, and this causes great confusion when you are debugging your application in Firebug.  Imagine you can see that customer.name has a value when you hit a breakpoint, but its not what you’re looking for.  Sensei developed a tactic to help verify that he’s not insane, and it works simply.  When in doubt, go the console in Firebug and access your observable via the ViewModel; so in our case you could issue:

vm.customer.name();

When name() doesn’t match your expectation you’ve most likely added a property with a typo.  Check with

vm.customer.name;

It sounds silly, but you can easily spend a half hour insisting that you’re doing the right thing, but you really confusing a property with a method.  Furthermore, observable arrays can also be a source of frustration:

// This is not the length of the observable array. It will always be zero!!!
vm.customers.length == 0;

// You get the length with this syntax
vm.customers().length;

Knock ’em inta tamarra, Rocky

Had Sensei known the two tips before starting he would have save a lot of time.  There are many others, and they are best described by Ryan Niemeyer in his post 10 things to know about Knockout from day one.  Read this post slowly.  It will save you a lot of headache.  You may familiar with jQuery and Javascript, but Knockout introduces subtle differences that will catch you off guard.  That’s not a bad thing, it’s just different than what you may be used to.  Ryan also makes great use of JS Fiddle and answers most of his StackOverflow questions by using examples.  Those examples are in many cases easier to learn from than the tutorial since the scope is narrower than the instruction that Steve Sanderson gives.  It really allows you play along as you learn.

More RavenDB Resources January 3, 2012

Posted by ActiveEngine Sensei in .Net Development, C#, New Techniques, Open Source, RavenDB.
Tags: , ,
3 comments

Daniel Lang has a great post regarding how to handle relations in RavenDB.  He emphasizes that a document database is vastly different from a relation database and illustrates various scenarios of do’s and don’ts.  Go read it now.

ApprovaFlow: Where We At? December 3, 2011

Posted by ActiveEngine Sensei in .Net, ActiveEngine, Approvaflow, ASP.Net, RavenDB, Workflow.
Tags: , , ,
add a comment

It’s been a while, and as usual Sensei has started something with such bravado and discovered that life offers more bluster and pounding than even he can anticipate.  Hopefully you haven’t given up on the series, ’cause Sensei hasn’t.  Hell, ApprovaFlow is constantly on the forefront, even though it appears that he’s taken a good powder.

Let’s recap our goals and talk about philosophy and direction.  In this long silence a few additional considerations have taken precedence, and this is a good opportunity to assess goals and re-align the development efforts.  In the end ApprovaFlow will:

 Model a workflow in a clear format that is readable by both developer and business user. One set of verbiage for all parties.  Discussed in Simple Workflows With ApprovaFlow and Stateless.

  Allow the state of a workflow to be peristed as an integer, string, etc.  Quickly fetch state of a workflow.  Discussed in Simple Workflows With ApprovaFlow and Stateless.

 Create pre and post processing methods that can enforce enforce rules or carry out actions when completing a workflow task.  Discussed inApprovaFlow:  Using the Pipe and Filter Pattern to Build a Workflow Processor

• Introduce new functionality while isolating the impact of the new changes. New components should not break old ones

• Communicate to the client with a standard set of objects. In other words, your solution domain will not change how the user interface will gather data from the user.

• Use one. aspx page to processes user input for any type of workflow.

• Provide ability to roll your own customizations to the front end or backend of your application.

The astute members of the audience will no doubt say “What about the technical objectives, like how are you going to store all the workflow data?  In flat files?  Will you give me alternatives for storage?  How will I create the workflows, by using Notepad?” Indeed, Sensei has pondered these issues as well and has accumulated a fair amount failed experiments with some being quite interesting.   Given time these little experiments may become posts as well, since there are interesting things to learn from these failures.

What ApprovaFlow Will Need To Provide:  Workflow Storage

The biggest issue is storage.  The point of using Stateless was that we wanted flexibility.  Recall that the state of our state machine can be represented with a mere integer or string.  Makes it pretty easy to store this in a database, or a document.  While you could map the Step and Workflow class to tables in SQL our domain is using JSON so it makes sense to gravitate to a storage solution that will easily support that format.  ApprovaFlow will use RavenDB as the document database, but will provide the opportunity for you to use a different solution if you wish.  You’ll find that RavenDB quite readily provides a document storage format for our workflows that is quite elegant.

As an aside, Sensei experimented with a great alternative to the NoSQL solutions called Sis0DB.  This open project provides you that ability to store you object graphs in SQL Server.  Time permitting Sensei will share some of his adventures with you regarding this neat project.

What ApprovaFlow Will Need to Provide:  Authorization of Actions

While Sensei was off in the weeds learning about RavenDB he discovered that Ayende created a fantastic mechanism for authorizing user actions on documents.  This authorization of activity can be a granular as denying / allowing updates to occur based on an operation.

Since we want to adhere to principles of flexibility the Authorization features will be implemented as a plug-in, so if you wish to roll your own mechanisms to govern workflow approvals you will be free to do so.

What ApprovaFlow Will Need to Provide:  Admin Tools

Yep.  Sensei is sick of using Notepad to create JSON documents as well.  We want to be able to create the states, the triggers and the target states and save.  We’ll want to assign the filters to specific states and save.  No more text fiddling. Period.  As Sensei is thinking about this, it seems that another pipeline can be created for administration.  Luckily we have a plug-in architecture so this should be rather straight forward.

Summing It All Up

These are really important things to consider, and as much as Sensei hates changing goals in mid stream the capabilities discussed above can make life much easier while implementing a workflow system.  In making the decision to use RavenDB the thought that “a storage solution should not shape the solution domain” kept raising its ugly maw.  But, so what.   We want to finish something, and admittedly this has been a challenge – just look at the lag between posts if you need a reminder.  If Sensei decided to include an IOC container just to remain “loosely” coupled to document storage we’ll get no where.  Would you really want to read those posts?  How boring.  Besides, Sensei doesn’t know how to do all that stuf – gonna stick to the stuff he thinks he knows.  Or at least the stuff he can fake.

ApprovaFlow: Create A Plugin System And Reduce Deployment Headaches June 29, 2011

Posted by ActiveEngine Sensei in .Net, ActiveEngine, Approvaflow, ASP.Net, Problem Solving, Workflow.
Tags: , , , ,
2 comments

This is the fourth in a series of posts for ApprovaFlow, an alternative to Windows Workflow written in C# and JSON.Net.  Source code for this post is here.

Last Time on ApprovaFlow

In the previous post we discussed how the Pipe and Filter pattern facilitated a robust mechanism for executing tasks prior and after a transition is completed by the workflow state machine.  This accomplished our third goal and to date we have completed:

Model a workflow in a clear format that is readable by both developer and business user. One set of verbiage for all parties.  Discussed in Simple Workflows With ApprovaFlow and Stateless.

•  Allow the state of a workflow to be peristed as an integer, string, etc.  Quickly fetch state of a workflow.  Discussed in Simple Workflows With ApprovaFlow and Stateless.

•  Create pre and post processing methods that can enforce enforce rules or carry out actions when completing a workflow task.  Discussed in ApprovaFlow:  Using the Pipe and Filter Pattern to Build a Workflow Processor

These goals remain:

• Introduce new functionality while isolating the impact of the new changes. New components should not break old ones

• Communicate to the client with a standard set of objects. In other words, your solution domain will not change how the user interface will gather data from the user.

• Use one. aspx page to processes user input for any type of workflow.

• Provide ability to roll your own customizations to the front end or backend of your application.

It’s the Small Changes After You Go Live That Upset You

The goal we’ll focus on next is Introduce new functionality while isolating the impact of the new changes. New components should not break old ones, as it’s the small upsetters that lurk around the corner that your users will think up that will keep you in the constant redeployment cycle. If we implement a plug-in system, then we can prevent the new features from breaking the current production system. Implementing these changes in isolation will lead to faster testing, validation and happier users.

We lucked out as our implementation of the Pipe And Filter pattern forced us to create objects with finite functionality.  If you recall each step in our workflow chain was implemented as a filter derived from FilterBase and this lends itself nicely to creating plug-ins.  The Pipe and Filter pattern forces us to have a filter for each unique action we wish to carry out.  To save data we have a SaveData filter, to validate that a user can supply a Trigger we have the ValidateUserTrigger, and so on.

“Great, Sensei, but aren’t we still constrained by the fact that we have to recompile and deploy any time we add new filters?  And, if I have to do that, why bother with the pattern in the first place?”

Well, we can easily reduce the need for re-deploying the application through the use of a plugin system where we read assemblies from a share and interrogate them by searching for a particular object type on application start up.  Each new feature will be a new filter.  This means you will be working with a small project that references ApprovaFlow to create new filters without disturbing the existing architecture.   We’ll also create a manifest of approved plug-ins so that we can control what is used and institute a little security since we wouldn’t want any plugin to be introduced surreptitiously.

Plug-in Implementation

The class FilterRegistry will perform the process of reading a share, fetching the object with type FilterBase, and register these components just like we do with our system components.  There are a few additions since the last version, as we now need to read and store the manifest for later comparison with the plug-ins.  The new method ReadManifest takes care of this new task:

private void ReadManifest()
{
  string manifestSource = ConfigurationManager.AppSettings["ManifestSource"].ToString();

  Enforce.That(string.IsNullOrEmpty(manifestSource) == false,
          "FilterRegistry.ReadManifest - ManifestSource can not be null");

  var fileInfo = new FileInfo(manifestSource);

  if (fileInfo.Exists == false)
  {
    throw new ApplicationException("RequestPromotion.Configure - File not found");
  }

  StreamReader sr = fileInfo.OpenText();
  string json = sr.ReadToEnd();
  sr.Close();

  this.approvedFilters = JsonConvert.DeserializeObject>>(json);
}

The manifest is merely a serialized list of FilterDefinitions. This is de-serialized into a list of approved filters.With the approved list the method LoadPlugin performs the action of reading the share and matching the FullName of the object type between the manifest entries and the methods in the assembly file:

public void LoadPlugIn(string source)
{
  Enforce.That(string.IsNullOrEmpty(source) == false,
             "PlugInLoader.Load - source can not be null");

  AppDomain appDomain = AppDomain.CurrentDomain;
  var assembly = Assembly.LoadFrom(source);

  var types = assembly.GetTypes().ToList();

  types.ForEach(type =>
  {
    var registerFilterDef = new FilterDefinition();

    //  Is type from assembly registered?
    registerFilterDef = this.approvedFilters.Where(app => app.TypeFullName == type.FullName)
                                   .SingleOrDefault();

    if (registerFilterDef != null)
    {
      object obj = Activator.CreateInstance(type);
      var filterDef = new FilterDefinition();
      filterDef.Name = obj.ToString();
      filterDef.FilterCategory = registerFilterDef.FilterCategory;
      filterDef.FilterType = type;
      filterDef.TypeFullName = type.FullName;
      filterDef.Filter = AddCreateFilter(filterDef);

      this.systemFilters.Add(filterDef);
     }
  });
}

That’s it. We can now control what assemblies are included in our plug-in system.  Later we’ll create a tool that will help us create the manifest so we do not have to managed it by hand.

What We Can Do with this New Functionality

Let’s turn to our sample workflow to see what possibilities we can develop.  The test CanPromoteRedShirtOffLandingParty from the class WorkflowScenarios displays the capability of our workflow.  First lets review our workflow scenario.  We have created a workflow for the Starship Enterprise to allow members of a landing party to request to be left out of the mission.  Basically there is only one way to get out of landing party duty and that is if Kirk says it’s okay.  Here are the workflow’s State, Trigger and Target State combinations:

State Trigger Target State
RequestPromotionForm Complete FirstOfficerReview
FirstOfficerReview RequestInfo RequestPromotionForm
FirstOfficerReview Deny PromotionDenied
FirstOfficerReview Approve CaptainApproval
CaptainApproval OfficerJustify FirstOfficerReview
CaptainApproval Deny PromotionDenied
CaptainApproval Approve PromotedOffLandingParty

Recalling the plots from Star Trek, there were times that the medical officer could declare the commanding officer unfit for duty. Since the Enterprise was originally equipped with our workflow, we want to make just a small addition – not a modification – and give McCoy the ability to allow a red shirt to opt out of the landing party duty.

Here’s where our plugin system comes in handy.  Instead of adding more states and or branches to our workflow we’ll check for certain conditions when Kirk makes his decisions, and execute actions.  In order to help out McCoy the following filter is created in a separate project:

public class CaptainUnfitForCommandFilter : FilterBase
{
  protected override Step Process(Step input)
  {
    if(input.CanProcess & input.State == "CaptainApproval")
    {
      bool kirkInfected = (bool)input.Parameters["KirkInfected"];

      if(kirkInfected & input.Answer == "Deny")
      {
        input.Parameters.Add("MedicalOverride", true);
        input.Parameters.Add("StarfleetEmail", true);
        input.ErrorList.Add("Medical Override of Command");
        input.CanProcess = false;
      }
    }

    return input;
  }
}

This plug-in is simple: check that the state is CaptainApproval and when the answer was “Deny” and Kirk has been infected, set the MedicalOverride flag and send Starfleet an email.

The class WorkflowScenarioTest.cs has the method CanAllowMcCoyToIssueUnfitForDuty() that demonstrates how the workflow will execute. We simply add the name of the plug-in to our list of post transition filters:

string postFilterNames = "MorePlugins.TransporterRepairFilter;Plugins.CaptainUnfitForCommandFilter;SaveDataFilter;";

This portion of code uses the plug-in:

//  Captain Kirt denies request, but McCoy issues unfit for command
parameters.Add("KirkInfected", true);

step.Answer = "Deny";
step.AnsweredBy = "Kirk";
step.Participants = "Kirk";
step.State = newState;

processor = new WorkflowProcessor(step, filterRegistry, workflow);
newState = processor.ConfigurePipeline(preFilterNames, postFilterNames)
  .ConfigureStateMachine()
  .ProcessAnswer()
  .GetCurrentState();

//  Medical override issued and email to Starfleet generated
bool medicalOverride = (bool)parameters["MedicalOverride"];
bool emailSent = (bool)parameters["StarfleetEmail"];

Assert.IsTrue(medicalOverride);
Assert.IsTrue(emailSent);

Now you don’t have to hesitate with paranoia each time you need introduce a variation into your workflows. No more small upsetters lurking around the corner. Plus you can deliver these changes faster to your biggest fan, your customer. Source code is here.   Run through the tests and experiment for your self.

DataTablePager Now Has Multi-Column Sort Capability For DataTables.Net February 9, 2011

Posted by ActiveEngine Sensei in .Net, ActiveEngine, Ajax, ASP.Net, C#, DataTables.Net, jQuery.
Tags: , , , , , , , , ,
21 comments

Some gifts just keep on giving, and many times things can just take on a momentum that grow beyond your expectation.  Bob Sherwood wrote to Sensei and pointed out that DataTables.net supports multiple column sorting.  All you do is hold down the shift key and click on any second or third column and DataTables will add that column to sort criteria.  “Well, how come it doesn’t work with the server side solution?”  Talk about the sound of one hand clapping.  How about that for a flub!  Sensei didn’t think of that!  Then panic set in – would this introduce new complexity to the DataTablePager solution, making it too difficult to maintain a clean implementation?  After some long thought it seemed that a solution could be neatly added.  Before reading, you should download the latest code to follow along.

How DataTables.Net Communicates Which Columns Are Involved in a Sort

If you recall, DataTables.Net uses a structure called aoData to communicate to the server what columns are needed, the page size, and whether a column is a data element or a client side custom column.  We covered that in the last DataTablePager post.  aoData also has a convention for sorting:

bSortColumn_X=ColumnPosition

In our example we are working with the following columns:

,Name,Agent,Center,,CenterId,DealAmount

where column 0 is a custom client side column, column 1 is Name (a mere data column), column 2 is Center (another data column), column 3 is a custom client side column, and the remaining columns are just data columns.

If we are sorting just by Name, then aoData will contain the following:

bSortColumn_0=1

When we wish to sort by Center, then by Name we get the following in aoData”

bSortColumn_0=2

bSortColumn_1=1

In other words, the first column we want to sort by is in position 2 (Center) and the second column(Name) is in position 1.  We’ll want to record this some where so that we can pass this to our order routine.  aoData passes all column information to us on the server, but we’ll have to parse through the columns and check to see if one or many of the columns is actually involved in a sort request and as we do we’ll need to preserve the order of that column of data in the sort.

SearchAndSortable Class to the Rescue

You’ll recall that we have a class called SearchAndSortable that defines how the column is used by the client.  Since we iterate over all the columns in aoData it makes sense that we should take this opportunity to see if any column is involved in a sort and store that information in SearchAndSortable as well.  The new code for the class looks like this:

public class SearchAndSortable
    {
        public string Name { get; set; }
        public int ColumnIndex { get; set; }
        public bool IsSearchable { get; set; }
        public bool IsSortable { get; set; }
        public PropertyInfo Property{ get; set; }
        public int SortOrder { get; set; }
        public bool IsCurrentlySorted { get; set; }
        public string SortDirection { get; set; }

        public SearchAndSortable(string name, int columnIndex, bool isSearchable,
                                bool isSortable)
        {
            this.Name = name;
            this.ColumnIndex = columnIndex;
            this.IsSearchable = isSearchable;
            this.IsSortable = IsSortable;
        }

        public SearchAndSortable() : this(string.Empty, 0, true, true) { }
    }

There are 3 new additions:

IsCurrentlySorted – is this column included in the sort request.

SortDirection – “asc” or “desc” for ascending and descending.

SortOrder – the order of the column in the sort request.  Is it the first or second column in a multicolumn sort.

As we walk through the column definitions, we’ll look to see if each column is involved in a sort and record what direction – ascending or descending – is required. From our previous post you’ll remember that the method PrepAOData is where we parse our column definitions. Here is the new code:

//  Sort columns
this.sortKeyPrefix = aoDataList.Where(x => x.Name.StartsWith(INDIVIDUAL_SORT_KEY_PREFIX))
                                            .Select(x => x.Value)
                                            .ToList();

//  Column list
var cols = aoDataList.Where(x => x.Name == "sColumns"
                                            & string.IsNullOrEmpty(x.Value) == false)
                                     .SingleOrDefault();

if(cols == null)
{
  this.columns = new List();
}
else
{
  this.columns = cols.Value
                       .Split(',')
                       .ToList();
}

//  What column is searchable and / or sortable
//  What properties from T is identified by the columns
var properties = typeof(T).GetProperties();
int i = 0;

//  Search and store all properties from T
this.columns.ForEach(col =>
{
  if (string.IsNullOrEmpty(col) == false)
  {
    var searchable = new SearchAndSortable(col, i, false, false);
    var searchItem = aoDataList.Where(x => x.Name == BSEARCHABLE + i.ToString())
                                     .ToList();
    searchable.IsSearchable = (searchItem[0].Value == "False") ? false : true;
    searchable.Property = properties.Where(x => x.Name == col)
                                                    .SingleOrDefault();

    searchAndSortables.Add(searchable);
  }

  i++;
});

//  Sort
searchAndSortables.ForEach(sortable => {
  var sort = aoDataList.Where(x => x.Name == BSORTABLE + sortable.ColumnIndex.ToString())
                                            .ToList();
  sortable.IsSortable = (sort[0].Value == "False") ? false : true;
                sortable.SortOrder = -1;

  //  Is this item amongst currently sorted columns?
  int order = 0;
  this.sortKeyPrefix.ForEach(keyPrefix => {
    if (sortable.ColumnIndex == Convert.ToInt32(keyPrefix))
    {
      sortable.IsCurrentlySorted = true;

      //  Is this the primary sort column or secondary?
      sortable.SortOrder = order;

     //  Ascending or Descending?
     var ascDesc = aoDataList.Where(x => x.Name == "sSortDir_" + order)
                                                    .SingleOrDefault();
     if(ascDesc != null)
     {
       sortable.SortDirection = ascDesc.Value;
     }
   }

   order++;
 });
});

To sum up, we’ll traverse all of the columns listed in sColumns. For each column we’ll grab the PorpertyInfo from our underlying object of type T. This gives only those properties that will be displayed in the grid on the client. If the column is marked as searchable, we indicate that by setting the IsSearchable property on the SearchAndSortable class.  This happens starting at line 28 through 43.

Next we need to determine what we can sort, and will traverse the new list of SearchAndSortables we created. DataTables will tell us what if the column can be sorted by with following convention:

bSortable_ColNumber = True

So if the column Center were to be “sortable” aoData would contain:

bSortable_1 = True

We record the sortable state as shown on line 49 in the code listing.

Now that we know whether we can sort on this column, we have to look through the sort request and see if the column is actually involved in a sort.  We do that by looking at what DataTables.Net sent to us from the client.  Again the convention is to send bSortColumn_0=1 to indicate that the first column for the sort in the second item listed in sColumns property.  aoData will contain many bSortColum’s so we’ll walk through each one and record the order that column should take in the sort.  That occurs at line 55 where we match the column index with the bSortColumn_x value.

We’ll also determine what the sort direction – ascending or descending – should be.  At line 63 we get the direction of the sort and record this value in the SearchAndSortable.

When the method PrepAOData is completed, we have a complete map of all columns and what columns are being sorted, as well as their respective sort direction.  All of this was sent to us from the client and we are storing this configuration for later use.

Performing the Sort

(Home stretch so play the song!!)

If you can picture what we have so far we just basically created a collection of column names, their respective PropertyInfo’s and have recorded which of these properties are involved in a sort.  At this stage we should be able to query this collection and get back those properties and the order that the sort applies.

You may already be aware that you can have a compound sort statement in LINQ with the following statement:

var sortedCustomers = customer.OrderBy(x => x.LastName)
                                           .ThenBy(x => x.FirstName);

The trick is to run through all the properties and create that compound statement. Remember when we recorded the position of the sort as an integer? This makes it easy for us to sort out the messy scenarios where the second column is the first column of a sort. SearchAndSortable.SortOrder takes care of this for us. Just get the data order by SortOrder in descending order and you’re good to go. So that code would look like the following:

var sorted = this.searchAndSortables.Where(x => x.IsCurrentlySorted == true)
                                     .OrderBy(x => x.SortOrder)
                                     .ToList();

sorted.ForEach(sort => {
             records = records.OrderBy(sort.Name, sort.SortDirection,
             (sort.SortOrder == 0) ? true : false);
});

On line 6 in the code above we are calling our extension method OrderBy in Extensions.cs. We pass the property name, the sort direction, and whether this is the first column of the sort. This last piece is important as it will create either “OrderBy” or the “ThenBy” for us. When it’s the first column, you guessed it we get “OrderBy”. Sensei found this magic on a StackOverflow post by Marc Gravell and others.

Here is the entire method ApplySort from DataTablePager.cs, and note how we still check for the initial display of the data grid and default to the first column that is sortable.

private IQueryable ApplySort(IQueryable records)
{
  var sorted = this.searchAndSortables.Where(x => x.IsCurrentlySorted == true)
                                                .OrderBy(x => x.SortOrder)
                                                .ToList();

  //  Are we at initialization of grid with no column selected?
  if (sorted.Count == 0)
  {
    string firstSortColumn = this.sortKeyPrefix.First();
    int firstColumn = int.Parse(firstSortColumn);

    string sortDirection = "asc";
    sortDirection = this.aoDataList.Where(x => x.Name == INDIVIDUAL_SORT_DIRECTION_KEY_PREFIX +                                                                    "0")
                                                    .Single()
                                                    .Value
                                                    .ToLower();

    if (string.IsNullOrEmpty(sortDirection))
    {
      sortDirection = "asc";
    }

    //  Initial display will set order to first column - column 0
    //  When column 0 is not sortable, find first column that is
    var sortable = this.searchAndSortables.Where(x => x.ColumnIndex == firstColumn)
                                                        .SingleOrDefault();
    if (sortable == null)
    {
      sortable = this.searchAndSortables.First(x => x.IsSortable);
    }

    return records.OrderBy(sortable.Name, sortDirection, true);
  }
  else
  {
      //  Traverse all columns selected for sort
      sorted.ForEach(sort => {
                             records = records.OrderBy(sort.Name, sort.SortDirection,
                            (sort.SortOrder == 0) ? true : false);
      });

    return records;
  }
}

It’s All in the Setup

Test it out. Hold down the shift key and select a second column and WHAMO – multiple column sorts! Hold down the shift key and click the same column twice and KAH-BLAMO multiple column sort with descending order on the second column!!!

The really cool thing is that our process on the server is being directed by DataTables.net on the client.  And even awseomer is that you have zero configuration on the server.  Most awesome-est is that this will work with all of your domain objects, because we have used generics we can apply this to any class in our domain.  So what are you doing to do with all that time you just got back?

Dynamically Select Columns with Server-Side Paging and Datatables.Net January 14, 2011

Posted by ActiveEngine Sensei in .Net, ActiveEngine, Ajax, ASP.Net, DataTables.Net, jQuery, JSON.Net, New Techniques, Problem Solving.
Tags: , , , , , ,
30 comments

Source code has been yet again updated!! Read about the changes in DataTablePager Now Has Multi-Column Sort Capability For DataTables.Net If you are new to DataTables.Net and Sensei’s paging solution and want to detailed study of how it works, work through this post first, then get the latest edition.  Note, code links in this post are to the first version.

The last episode of server-side paging with DataTablerPager for DataTables.Net we reviewed the basics of a server-side solution that paged records and returned results in the multiples as specified by DataTables.Net.  You will want to have read that post before preceding here.  The older version of the source is included in that post as well as this will help get you acclimated.  The following capabilities were reviewed:

  • The solution used generics and could work with any collection of IQueryable.  In short any of your classes from you domain solution  could be used.
  • Filtering capability across all properties was provided.  This included partial word matching, regardless of case.
  • Ordering of result set was in response to the column clicked on the client’s DataTables grid.

DataTablePager Enhancements

This past month Sensei has added new capabilities to the DataTablePager class that makes it an even better fit for use with DataTables.Net.  The new features are:

  • Dynamically select the columns from the properties of your class based on the column definitions supplied by DataTables.Net!!!
  • Exclude columns from sort or search based on configuration by DataTables.Net
  • Mix columns from your class properties with client-side only column definitions; e.g. create a column with <a href>’s that do not interfere with filtering, sorting, or other processing.

Before we jump into the nitty-gritty details let’s review how DataTables.Net allows you to control a column’s interaction with a data grid.  Grab the new source code to best follow along.

DataTables.Net Column Definition

You would think that there would be quite a few steps to keep your server-side data paging solution in concert with a client side implementation, and that would mean customization for each page.   DataTables.Net provides you with fine control over what your columns will do once displayed in a data grid.  Great, but does that mean a lot of configuration on the server side of the equation?  As well soon see, no, it doesn’t.  What is done on the client for configuration will be that you need to do.

The structure aoColumnDefs is the convention we use for column configuration.  From the DataTables.Net site:

aoColumnDefs: This array allows you to target a specific column, multiple columns, or all columns, using the aTargets property of each object in the array (please note that aoColumnDefs was introduced in DataTables 1.7). This allows great flexibility when creating tables, as the aoColumnDefs arrays can be of any length, targeting the columns you specifically want. The aTargets property is an array to target one of many columns and each element in it can be:

  • a string – class name will be matched on the TH for the column
  • 0 or a positive integer – column index counting from the left
  • a negative integer – column index counting from the right
  • the string “_all” – all columns (i.e. assign a default)

So in order for you to include columns in a sort you configure in this manner:

/* Using aoColumnDefs */
$(document).ready(function() {
	$('#example').dataTable( {
		"aoColumnDefs": [
			{ "bSortable": false, "aTargets": [ 0 ] }
		] } );
} );

} );

In other words we are defining that the first column – column 0 – will not be included in the sorting operations.  When you review the columns options you’ll see you have options for applying css classes to multiple columns, can include a column in filtering, can supply custom rendering of a column, and much more.

In the example that we’ll use for the rest of the post we are going to provide the following capability for a data grid:

  1. The first column – column 0 – will be an action column with a hyperlink, and we will want to exclude it form sort and filtering functions.
  2. Only display a subset of the properties from a class.  Each of these columns should be sortable and filterable.
  3. Maintain the ability to chunk the result set in the multiples specified by DataTables.Net; that is, multiples of 10, 50, and 100.

Here is the configuration from the aspx page SpecifyColumns.aspx:

"aoColumnDefs" : [
   {"fnRender" : function(oObj){
      return "<a href="&quot;center.aspx?centerid=&quot;">Edit</a>";
   },
     "bSortable" : false,
     "aTargets" : [0]},
   {"sName" : "Name",
     "bSearchable" : true,
     "aTargets": [1]},
   {"sName" : "Agent",
    "bSearchable" : true,
    "bSortable" : true,
    "aTargets" : [2]
   },
   {"sName" : "Center", "aTargets": [3]},
   {"fnRender" : function(oObj){
            return "2nd Action List";
         },
     "bSortable" : false,
     "aTargets" : [4]},
   {"sName" : "CenterId", "bVisible" : false, "aTargets" : [5]},
   {"sName" : "DealAmount", "aTargets" : [6]}
]
  1. Column 0 is our custom column – do not sort or search on this content.  Look at oObj.aData[4] – this is a column that we’ll return but not display.  It’s referred so by the position in the data array that DataTables.Net expects back from the server.
  2. Columns 1 – 3 are data and can be sorted.  Note the use of “sName”.  This will be included in a named column list that corresponds to the source property from our class.  This will be very important later on for us, as it allows us to query our data and return it in any order to DataTables.Net.  DataTables will figure out what to do with it before it renders.
  3. Threw in another custom column.  Again, no sort or search, but we’ll see how this affects the server side implementation later on.  Hint – there’s no sName used here.
  4. Another data column.

To recap, we want to be able to define what data we need to display and how we want to interact with that data by only instructing DataTables.Net what to do.  We’re going to be lazy, and not do anything else – the class DataTablePager will respond to the instructions that DataTables.Net supplies, and that’s it.  We’ll review how to do this next.  Sensei thinks you’ll really dig it.

DataTablePager Class Handles your Client Side Requests

If you recall, DataTables.Net communicates to the server via the structure aoData.  Here is the summary of the parameters.  One additional parameter that we’ll need to parse is the sColumns parameter, and it will contain the names and order of the columns that DataTables.Net is rendering.  For our example, we’ll get the following list of columns if we were to debug on the server:

,Name,Agent,Center,,CenterId,DealAmount

These are all the columns we named with sName, plus a place holder for those custom columns that not found in our class.  This has several implications.  For one, it will mean that we will no longer be able to simply use reflection to get at our properties, filter them and send them back down to the client.  The client is now expecting an array where each row will have 7 things, 5 of which are named and two place holders for items that the client wants to reserve for itself.  Hence the convention of passing an empty item in the delimited string as shown above.

It will also mean that we’ll have to separate the columns that we can filter or sort.  Again this is the reason for leaving the custom column names blank.  In other words, we’ll have to keep track of the items that we can search and sort.  We’ll do this with a class called SearchAndSortable:

public class SearchAndSortable
    {
        public string Name { get; set; }
        public int ColumnIndex { get; set; }
        public bool IsSearchable { get; set; }
        public bool IsSortable { get; set; }
        public PropertyInfo Property{ get; set; }

        public SearchAndSortable(string name, int columnIndex, bool isSearchable, bool isSortable)
        {
            this.Name = name;
            this.ColumnIndex = columnIndex;
            this.IsSearchable = isSearchable;
            this.IsSortable = IsSortable;
        }

        public SearchAndSortable() : this(string.Empty, 0, true, true) { }
    }

This will summarize what we’re doing with our properties.   The property ColumnIndex will record the position in sColumn where our column occurs.  Since we’ll need access to the actual properties themselves we’ll store these in the SearchAndSortable as well so that we can reduce the number of calls that use reflection. DataTablePager uses a List of SortAndSearchables to track what’s going on.  We fill this list in the method PrepAOData()

//  What column is searchable and / or sortable
            //  What properties from T is identified by the columns
            var properties = typeof(T).GetProperties();
            int i = 0;

            //  Search and store all properties from T
            this.columns.ForEach(col =>
            {
                if (string.IsNullOrEmpty(col) == false)
                {
                    var searchable = new SearchAndSortable(col, i, false, false);
                    var searchItem = aoDataList.Where(x => x.Name == BSEARCHABLE + i.ToString())
                                     .ToList();
                    searchable.IsSearchable = (searchItem[0].Value == "False") ? false : true;
                    searchable.Property = properties.Where(x => x.Name == col)
                                                    .SingleOrDefault();

                    searchAndSortables.Add(searchable);
                }

                i++;
            });

            //  Sort
            searchAndSortables.ForEach(sortable => {
                var sort = aoDataList.Where(x => x.Name == BSORTABLE + sortable.ColumnIndex.ToString())
                                            .ToList();
                sortable.IsSortable = (sort[0].Value == "False") ? false : true;
            });

We’ll get the properties from our class. Next we’ll traverse the columns and match the property names with the names of the columns. When there is a match, we need to query aoData and get the column search and sort definitions based on the ordinal position of the column in the sColumns variable. DataTables.Net convention for communicating this is the form of:

bSortable_ + column index => “bSortable_1” or “bSearchable_2”

We take care of that with this line of code:

var searchItem = aoDataList.Where(x => x.Name == BSEARCHABLE +
                                     i.ToString())
                                     .ToList();
searchable.IsSearchable = (searchItem[0].Value == "False") ? false : true;

Now we go through the list of properties again but this time determine if we should sort any of the columns. That happens in the section //Sort. In the end we have a list of properties that corresponds with the columns DataTables.Net has requested, and we have defined if the property can be search (filtered) or sorted.

For filtering DataTablePager recall that we use the method GenericSearchFilter().  The only alteration here is that we only will add the properties to our query that are defined as searcable:

//  Create a list of searchable properties
            var filterProperties = this.searchAndSortables.Where(x =>
                                        x.IsSearchable)
                                          .Select(x => x.Property)
                                          .ToList();

The rest of the method is unaltered from the prior version. Pretty cool!! Again, we’ll only get the properties that we declared as legal for filtering. We’ve also eliminated any chance of mixing a custom column in with our properties because we did not supply an sName in our configuration.

The method ApplySort() required one change. On the initial load of DataTable.Net, the client will pass up the request to sort on column 0 even though you may have excluded it. When that is the case, we’ll just look for the first column that is sortable and order by that column.

//  Initial display will set order to first column - column 0
//  When column 0 is not sortable, find first column that is
var sortable = this.searchAndSortables.Where(x => x.ColumnIndex ==
                                         firstColumn)
                              .SingleOrDefault();
if(sortable == null)
{
   sortable = this.searchAndSortables.First(x => x.IsSortable);
}

return records.OrderBy(sortable.Name, sortDirection, true);

After we have filtered and sorted the data set we can finally select the only those properties that we want to send to the client.  Recall that we have parsed a variable sColumns that tells what columns are expected.  We’ll pass these names onto extension method PropertiesToList().  This method will only serialize the property if the column is include, and since we have already paired down our data set as a result of our query and paging, there is very little performance impact.  Here is the new PropertiesToList method:

public static ListPropertiesToList(this T obj, List propertyNames)
{
   var propertyList = new List();
   var properties = typeof(T).GetProperties();
   var props = new List();

   //  Find all "" in propertyNames and insert empty value into list at
   //  corresponding position
   var blankIndexes = new List();
   int i = 0;

   //  Select and order filterProperties.  Record index position where there is
   //  no property
   propertyNames.ForEach(name =>
   {
      var property = properties.Where(prop => prop.Name == name.Trim())
         .SingleOrDefault();

      if(property == null)
      {
         blankIndexes.Add(new NameValuePair(name, i));
      }
      else
      {
         props.Add(properties.Where(prop => prop.Name == name.Trim())
                                    .SingleOrDefault());
      }
      i++;
   });

   propertyList = props.Select(prop => (prop.GetValue(obj, new object[0]) ?? string.Empty).ToString())
                                        .ToList();

   //  Add "" to List as client expects blank value in array
   blankIndexes.ForEach(index =>; {
      propertyList.Insert(index.Value, string.Empty);
   });

   return propertyList;
}

You might ask why not just pass in the list of SearchAndSortTable and avoid using reflection again. You could, but remember at this point we have reduced the number of items to the page size of 10, 50 or 100 rows, so your reflection calls will not have that great an impact. Also you should consider whether you want to simply have a function that will select only those properties that you need. Using SearchAndSortable would narrow the scope of utility, as you can use this method in other areas other than prepping data for DataTables.Net.

Now It’s Your Turn

That’s it.  Play with the page named SpecifyColumns.aspx.  You should be able to add and remove columns in the DataTable.Net configuration and they will just work.  This will mean, however, that you’ll have to always define your columns in your aspx page.  But since we worked really hard the first time around, DataTablePager will still be able to create paged data sets for any class in your domain.

Source code is here.  Enjoy.

%d bloggers like this: