ActiveEngine

Getting to 11 February 13, 2010

In the past Sensei has written insane tomes regarding time travel and how your best intentions really get you no where. The story today is about getting to 11, which as Nigel says is one more than ten, putting you over the top.  Consider for a moment the times that you really think you’re like this guy to the right.  Yep, you think you have a Martin Fowler sized audience when you are coding.  The scientists of the future will study my code and say “Here, this is the start of the great insight.  How interesting.”  In reality you are like Spinal Tap, unaware of how absurd you can be.  Code too complex, but it goes to 11!   Most blokes keep it at 10, but then you need to put it over the top take it up a notch.  That extra notch.  That’s 11.

Here’s a thought – what about 6?  Is it viable?  Can you be flexible by doing a 6, just good enough to not paint yourself into a corner?  “Perfection is a process, viable is an end state.”  As a developer you may not be able to judge what 6 is.  If you’re in tune with your fan base you’ll know but that can only come from wisdom born out of great mistakes.   For those of you who study Budo you may recall the concept of short and long and how relative scale can shift your advantage.  Your opponent may have a sword and you only a dagger.  Short and long makes a big difference, but you can alter that equation with small maneuver.  Once you’re inside and beyond the sword’s curring range you have the advantage, as your dagger is now long enough to finish the skirmish.  Change the scale.

Years back Sensei was given the task of reducing shelf space utilized by paper by 25%.  The CFO arrived at this goal via scientific method.  It was scientific since at Sensei’s company if you don’t do what the CFO said it is axiomatic that you were in deep doo-doo.  Laws of hierarchy and all.  Now imagine rooms filled with documents related to contractors, account-receiveables, human resources, legal contracts, project management etc.  Yah, DISPARATE is the word.  Not meta data, just a meta-mess.

Now in the best of all worlds where you need to get to 11, you would have time to survey all document types and refine each attribute set before you design your system for document categorization.  This foundation becomes your data model in a database and many would claim that you should a create data table per document type to house the varying number of attributes. But you have 2 million sheets of paper to scan and in 12 months re-construction at your offices begin so you need to be able to walk into a room and quickly categorize all documents, throw them into boxes, scan them, and automatically assign the meta data to the document and store the thing.  Oh, and if you miss a document type or need more attributes you don’t want to go back to your database, add or modify a table, re-gen your data access layer, add the attribute to your screen all before your adjust your categorization.  And remember, you need to ship out 80 to 100 boxes every 2 weeks so you need to keep the data entry flowing.  Finally, you are told that some projects can have up to 50 different types of documents, but no one is sure to what degree the project documentation is complete so the number of document types per project is not known and NOBODY HAS THE TIME TO GO THROUGH THE SHELVES AND CREATE DOCUMENT TYPES BEFORE ANY DATA ENTRY IS POSSIBLE!

Play the song, ’cause it adds to the excitement!!

Several key decisions solved this mess, and the solution was simple enough that temps could walk into a room categorize and pack documents into boxes for scanning.  The error rate ranged between 1 – 5% per department.  These were not solutions cranked up to 11, they were 6’s:

• No change to database schema or screens will be made, ever.  A document was modeled with four tables with a base Document table, Document- Type table, Document-Attribute table that contained all attributes per Document-Type and finally a Document-Attribute-Value table where the meta data was stored.  This way each Document-Type could be be created with simple data entry.  One data entry screen that could create data controls on the fly per attribute type developed.
• Each document shall have a bar-coded coversheet.  Nothing gets scanned without meta data.  EVV-ARR.
• Import data from existing systems. The meta data for your documents resides in many of your accounting, job cost, and budget systems.  Once document types are known, dumping data from accounts receivable and / or accounts payable allows your to assemble thousands of cover-sheets for all invoices.  Quite literally you create a stack of paper for all possible types of invoices for all accounts, walk into a room, pull documents of the shelf, attach coversheets, and keep the sheets you didn’t use.  Now, since the unused sheets have a bar-code, run these through you bar-code reader and create delete records for what you didn’t use.  Now you have a complete accurate manifest of what was on the shelf and what was packed away.  When the scanned images come back you can inspect them against the manifest.
• People can work better with paper.  As stated in the last bullet point, creating all possible types of document coversheets per account or project and printing them allows you to quickly categorize all documents.  With minimal or no data entry and a stack of coversheets, anyone now can go through shelves and associate the coversheets with the appropriate documents.  In other words, the subject matter experts have a tangible, traceable system that they can hand off and supervise someone who can do the grunt work.  Not sure where you finished with your categorization?  Just look at your stack of coversheets.  Want to inspect accuracy, grab a document and compare it to the categories printed on the coversheet.

What?  Process management with paper ?  That sucks!  No it really doesn’t.  You see, a 6 to you really is an eleventy-one for your user community who is really busy.  Yep, you have to be smart with your database design by focusing on one key area and that’s it.  The rest  of the effort is imports with SSIS packages, CSV files, and printed coversheets.  But it’s easy for the users to use paper, and that keeps a flow going.  2 million sheets of paper scanned in a year.  Maybe a 6 isn’t all that bad after all.