jump to navigation

ApprovaFlow: A Quick Primer On RavenDB December 15, 2011

Posted by ActiveEngine Sensei in .Net, .Net Development, ActiveEngine, Approvaflow, ASP.Net, Business Processes, JSON.Net, Open Source, RavenDB, Workflow.
Tags: , , , , ,
2 comments

This post focuses on getting started with RavenDB, so we’ll set aside our focus on workflows for a bit.  It’s included in the ApprovaFlow series as it is an important part of the workflow framework we’re building.  To follow along you might want to get the source code.

RavenDB is a document database that provides a flexible means for storing object graphs.  As you’ll see a document database presents you with a different set of challenges than you are normally presented when using a traditional relational database.

The storage “unit” in RavenDB is a schema-less JSON document.  This takes the form of:  Because you are working with documents you now have the flexibility to define documents differently; that is, you can support variations to your data without have to re-craft your data model each time you want to add a new property to a class.  You can adopt a “star” pattern for SQL as depicted here, but querying can become difficult.  Raven excels in this situation and one such sweet spot is:

Dynamic Entities, such as user-customizable entities, entities with a large number of optional fields, etc. – Raven’s schema free nature means that you don’t have to fight a relational model to implement it.

Installing and Running RavenDB

The compiled binaries are easy to install.  Download the latest build and extract the files to a share.  Note that in order to run the console you are required to install Silverlight.  To start the server, navigate to the folder[] and double click “Start.cmd”.  You will see a screen similar to this one once the server is up and running:

The console will launch it self and will resemble this:

How To Start Developing

In Visual Studio, reference Raven with Raven.Client.Lightweight.  For CRUD operations and querying this will be all that you will need.

First you will need to connect to the document store.  It is recommended that you do this once per application.  That is accomplished with

var documentStore = new DocumentStore {Url = "http://localhost:8080"};
documentStore.Initialize();

Procedures are carried out using the Unit of Work pattern, and in general you will be using these type of blocks:

using(var session = documentStore.OpenSession())
{
   //... Do some work
}

RavenDB will work with Plain Old C# Objects and only requires an Id property of type string.  An identity key is generated for Id during this session.  If were were to create multiple steps we would have identities created in succession.  A full discussion of the alternatives to the Id property is here.

Creating a document from your POCOs’ object graphs is very straight forward:

public class Person
{
    public string FirstName { get; set; }
public string LastName { get; set; }
public string Id { get; set; }
public int DepartmentId { get; set; }
    // ...
}

var person = new Person();

using(var session = documentStore.OpenSession())
{
   session.Store(person);
   session.SaveChanges();
}

Fetching a document can be accomplished in two manners:  by Id or with a LINQ query.  Here’s how to get a document by id:

string person = "Person/1";  //  Raven will have auto-generated a value for us.
using(var session = documentStore.OpenSession())
{
   var fetchedPerson = session.Load<Person>(personId);
   //Do some more work
}

You’ll note that there is no casting or conversion required as Raven will determine the object type and populate the properties for you.

There are naturally cases where you want to query for documents based on attributes other than the Id. Best practices guides that we should create static indexes on our documents as these will offer the best performance. RavenDB also has a dynamic index feature that learns from queries fired at the server and over time these dynamic indexes are memorialized.

For your first bout with RavenDB you can simply query the documents with LINQ.   The test code takes advantage of the dynamic feature.  Later you will want to create indexes based on how you most likely will retrieve the documents.  This is different that a traditional RDMS solution, where the data is optimized for querying.  A document database is NOT.

Continuing with our example of Person documents we would use:

int departmentId = 139;

using(var session = documentStore.OpenSession())
{
   var people = session.Query<Person>()
                          .Where(x => x.DepartmentId == departmentId)
                          .ToList();
}

In the source code for this post there are more examples of querying.

Debugging, Troubleshooting and Dealing with Frustration

Given that this is something new and an open source project you may find yourself searching for help and more guidelines.  One thing to avail yourself of while troubleshooting is the fact that RavenDB has REST interface and you can validate your assumptions – or worse, confirm your errors – by using curl from the command line.  For example, to create a document via http you issue:

curl -X POST http://localhost:8080/docs -d "{ FirstName: 'Bob', LastName: 'Smith', Address: '5 Elm St' }"

Each action that takes place on the RavenDB server is displayed in a log on the server console app.  Sensei had to resort to this technique when troubleshooting some issues when he first started.  This StackOverflow question details the travails.

Another area that threw Sensei for a loop at first was the nature of the RavenDB writing and maintaining indexes.  In short, indexing is a background process, and Raven is designed to be “eventually consistent”.  That means that there can be a latency between when a change is submitted, saved, and indexed in the repository so that it can be fetched via queries.  When running tests from NUnit this code did not operate as expected, yet the console reported that the document was created:

session.Store(teamMember);

int posttestCount = session.Query<TeamMember>()
                .Count();

According to the documentation you can overcome this inconsistency by declaring that you are willing to wait until RavenDB has completed its current write operation.   This code will get you the expected results:

int posttestCount = session.Query<TeamMember>()
              .Customize(x => x.WaitForNonStaleResults())
              .Count();

Depending on the number of tests you write you may wish to run RavenDB in Embedded mode for faster results.  This might prove useful for automated testing and builds.  The source code provided in this post does NOT use embedded mode; rather, you have need your server running as this gives you the opportunity to inspect documents and acclimate yourself to the database.

There is much more that you can do with RavenDB, such as creating indexes across documents, assign security to individual documents, and much more.  This primer should be enough to get you started.  Next post we’ll see how RavenDB will fit into the ApprovaFlow framework.  Grab the source, play around and get ready for the next exciting episode.

 

ApprovaFlow: Where We At? December 3, 2011

Posted by ActiveEngine Sensei in .Net, ActiveEngine, Approvaflow, ASP.Net, RavenDB, Workflow.
Tags: , , ,
add a comment

It’s been a while, and as usual Sensei has started something with such bravado and discovered that life offers more bluster and pounding than even he can anticipate.  Hopefully you haven’t given up on the series, ’cause Sensei hasn’t.  Hell, ApprovaFlow is constantly on the forefront, even though it appears that he’s taken a good powder.

Let’s recap our goals and talk about philosophy and direction.  In this long silence a few additional considerations have taken precedence, and this is a good opportunity to assess goals and re-align the development efforts.  In the end ApprovaFlow will:

 Model a workflow in a clear format that is readable by both developer and business user. One set of verbiage for all parties.  Discussed in Simple Workflows With ApprovaFlow and Stateless.

  Allow the state of a workflow to be peristed as an integer, string, etc.  Quickly fetch state of a workflow.  Discussed in Simple Workflows With ApprovaFlow and Stateless.

 Create pre and post processing methods that can enforce enforce rules or carry out actions when completing a workflow task.  Discussed inApprovaFlow:  Using the Pipe and Filter Pattern to Build a Workflow Processor

• Introduce new functionality while isolating the impact of the new changes. New components should not break old ones

• Communicate to the client with a standard set of objects. In other words, your solution domain will not change how the user interface will gather data from the user.

• Use one. aspx page to processes user input for any type of workflow.

• Provide ability to roll your own customizations to the front end or backend of your application.

The astute members of the audience will no doubt say “What about the technical objectives, like how are you going to store all the workflow data?  In flat files?  Will you give me alternatives for storage?  How will I create the workflows, by using Notepad?” Indeed, Sensei has pondered these issues as well and has accumulated a fair amount failed experiments with some being quite interesting.   Given time these little experiments may become posts as well, since there are interesting things to learn from these failures.

What ApprovaFlow Will Need To Provide:  Workflow Storage

The biggest issue is storage.  The point of using Stateless was that we wanted flexibility.  Recall that the state of our state machine can be represented with a mere integer or string.  Makes it pretty easy to store this in a database, or a document.  While you could map the Step and Workflow class to tables in SQL our domain is using JSON so it makes sense to gravitate to a storage solution that will easily support that format.  ApprovaFlow will use RavenDB as the document database, but will provide the opportunity for you to use a different solution if you wish.  You’ll find that RavenDB quite readily provides a document storage format for our workflows that is quite elegant.

As an aside, Sensei experimented with a great alternative to the NoSQL solutions called Sis0DB.  This open project provides you that ability to store you object graphs in SQL Server.  Time permitting Sensei will share some of his adventures with you regarding this neat project.

What ApprovaFlow Will Need to Provide:  Authorization of Actions

While Sensei was off in the weeds learning about RavenDB he discovered that Ayende created a fantastic mechanism for authorizing user actions on documents.  This authorization of activity can be a granular as denying / allowing updates to occur based on an operation.

Since we want to adhere to principles of flexibility the Authorization features will be implemented as a plug-in, so if you wish to roll your own mechanisms to govern workflow approvals you will be free to do so.

What ApprovaFlow Will Need to Provide:  Admin Tools

Yep.  Sensei is sick of using Notepad to create JSON documents as well.  We want to be able to create the states, the triggers and the target states and save.  We’ll want to assign the filters to specific states and save.  No more text fiddling. Period.  As Sensei is thinking about this, it seems that another pipeline can be created for administration.  Luckily we have a plug-in architecture so this should be rather straight forward.

Summing It All Up

These are really important things to consider, and as much as Sensei hates changing goals in mid stream the capabilities discussed above can make life much easier while implementing a workflow system.  In making the decision to use RavenDB the thought that “a storage solution should not shape the solution domain” kept raising its ugly maw.  But, so what.   We want to finish something, and admittedly this has been a challenge – just look at the lag between posts if you need a reminder.  If Sensei decided to include an IOC container just to remain “loosely” coupled to document storage we’ll get no where.  Would you really want to read those posts?  How boring.  Besides, Sensei doesn’t know how to do all that stuf – gonna stick to the stuff he thinks he knows.  Or at least the stuff he can fake.