To be or not to be (Agile)

Sorry , I haven't been posting a lot of stuff as much as I wanted to, I have recently been assigned the responsibility of managing a small project team. The last month or so has mostly been spent in a lot of meetings and team coordination.

So far from what Ive experienced, project management is all about making plans, updating them and meetings, and some more meetings. I'm not really sure at the moment if I want to continue this path fo project management or stay towards more on the architecture side.

I'm currently leading an internal project, so there is no requirement for costing or client meetings etc. I can classify project management practices for software in two major categories

1. Plan Driven Management
   
2. Agile Development Methods

I have been doing a bit of reading on the internet on different forums and article based sites for better management practices and which yields more productivity. One of the sites is techrepublic, a great site on general IT related articles and whitepapers

Plan Driven Management:

Plan driven management is the classic form of project management, in which you have one project manager who usually creates an upper level plan within the first 1-2 weeks of the start of the project, allocates resources and maintains the cost of the project.

The project manager is responsible for defining timeliness of the project from top to bottom to the very detailed pieces required

Agile Development Methods:

There are a number of methodologies available for agile development, Agile methodologies welcome change and focus more on the people and team effort rather than processes or plan management

I am most familiar with the SCRUM, Ive take two presentations on it, one during my MS and one during the PDC

In Agile development methodologies there is no single person responsible for complete planning, usually the complete plan is broken down into iterations and the team itself is responsible for further deciding who will do which task in order to complete the iteration.

Agile management requires that the initial level plan only define the upper level requirements, the detailed planning is done just in time. All agile methods focus on customer satisfaction, there also exists an agile manifesto, which consists of the underlying principles for all agile methods

more information can be found in these links:

Traditional Plan Driven Project Management
Agile Project Management
Agile Software Development

DSL experience in Karachi

For those of you reading this post from within Pakistan know the problems associated with getting a broadband connection. A lot of services do not provide QoS (Quality of service) or any affordable package. Recently I got a shared DSL home 256k unlimited download package from maxcom. So far I have been using it for almost 2 months without any issues, check out the speedtest result for my connection:
speedtest result

The experience so far with maxcom has been really great, the installation was free and service downtime is virtually zero.

Some providers are providing download speeds upto 2 Mbps via ADSL connection, but almost all of them are volume based. Unlimited packages cross the affordibility line for the average Pakistani.

But the prospects are good, like the launch of country wide Wimax solution by wateen telecom, the prices should come within affordable reach even for 2Mbps connections

2nd Day at PDC 07 , Stephen Forte to the rescue!

Thanks to stephen forte for putting some life back into PDC 07!, I feel the problem here is maturity of professionals. Our industry is still young and people are still new to concepts like application architecture and management processes required to deliver a successful solution. As such, the local presenters either dont have the presentation skills or the required experience to engage the audience in an interesting case study.

Otherwise the mismanagement from the PC Hotel management was slightly better than the first day, you can download all the slides for topics presented by stephen from here

My personal favorites were database design patterns and Scalable applications. I really wish Clement Vasters would come and visit sometime. I took some of his sessions at PDC 05', they were a real mind opener.

Regex Fundamentals

I went over this site over the weekend to brush up my skills on regular expressions, this is probably the one of the best and only site you'll need, it has basics, quick starts and detailed internal working of the regex engines

For people new to Regular Expressions or regex, it provides a powerful functionality to match patterns in text, it might seem confusing at first but once you get the hang of it its not that hard (but I think the testing data would take some thought!)

as I work on .NET, the framework provides you classes in the namespace System.Text.RegularExpressions

http://www.regular-expressions.info/


I'll try to post some examples later

First day at pakistan developers conference 2007

As some of you might know, PDC 2007 is happening in karachi on 19th and 20th June, today happens to be the first day of the event. Frankly, i was looking forward to some of the topics mentioned in the agenda here

My first was PDC 2005, which was really quite good with speakers like Rafal Lucaweicki and Clemens Vasters, at that time VS 2005 and SQL Server 2005 was to be released, so there was tons to learn from it. The list of speakers was amazing none the less and there were topics on domain specific modeling, sql server reporting services and even Avalon. It was truly inspiring, after that I kept following blogs of some of the technical leaders in the market.

Unfortunately, that was probably the last good PDC there ever was, I get the feeling that PDC has been going downhill ever since, the number of professional speakers are minimized and so is the list of topics.

Today I attended the first day at PDC 07 arranged at PC hotel, It started off by waiting for the chief guest IT Minister Leghari to arrive, which in keeping with tradations here at pakistan arrived a full hour late. eventually he arrived and the ceremony was kicked off with a speech from the minister and plaque award ceremony.

Plaques were given to all sorts of people, including a guy who wrote the new urdu dictionary and a bunch of other people who's achievements were not explained and other irrelevants from the developers conference. Probably IT minister thought it would be a good idea to get rid of all pending plaque presentations at the PDC 07 event, may it be relevant to the developer conference or not

Finally we got to the tea break an hour late, on our way back we were handed an updated schedule. On close examination of the schedule, the session on Architecture "decoupling business logic from services" which I was planning to take was removed from schedule without any explanation!

we decided to take Goksin Bakirs presentation on developing with windows Vista enchancements, the presentation was nice but it was an overview level, pretty much of which I already knew

then we had the (mis)managed lunch. One of my friends had to go on a hunger strike lol!, cuz he had'nt filled his plate enough the first time, after the food as gone the waiters refused to fill it up. By the time people moved to desert, all the spoons had disappeared

after that we took the second session on "using LINQ to retrieve business data" by Zia khan. Zia khan is more of a business entrepreneur than a tech guy, but he gave enough of an overview to get started. Again there wernt any live sessions or demo's , just some architecture slides and some code in slides. You would be better off by downloading a quickstart tutorial on LINQ

the last session we took was "using the right technology : Bizztalk, SQL Server, Sharepoint etc" by Naveed bajwa, the guy was a pretty horizontal tech guy, but the presentation could use some work. He explained some basics and discussed certain scenarios and asked the audience what technology can be used under the scenarios. Unfortunately, the scenarios were bland 2 liner requirements statement and without depth except the last one. It was hardly a topic for PDC, he could have been better off by writing a 2 page article on it

I really hope that tomorrows sessions are better, otherwise if we keep getting useless topics and inexperienced MVP speakers, PDC 07 would be loosing the last of its professional audience. Ever wonder why we have a Professional Developers Conference in the world all over and here its called Pakistan Developer Conference ??

The cheat sheet page

Jack daniel's website has a cheat sheet section, consisting of printable summarized commonly used methods for a number of tools and technologies including html, MySql, regular expressions and more.

I was looking for something on regular expressions when I came across this

click here to access his cheat sheet section

These are handy printouts that would help in any situation, I usually use one for javascript reference which can be found here its from the book Javascript Bible, it has save me many hours of unproductive searching

Karachi load shading reaches new heights

Karachi load shading has reached new heights, KESC is proabably in Guinness Book of world records by now for the longest load shading hours, yesterday we had load shading for about 7 hours. This is really bringing things to halt.


Checkout this image posted on karachi metblog
Good thing our office has its own power supply

Reverse Ajax

A couple of months back, i had posted simple AJAX tutorials on how to initiate simple AJAX calls and get the response from the server. Today I stumbled upon an article on reverse AJAX, click here to view the original post.

Reverse AJAX is actually an implementation (seems more of a marketing buzz word to me) rather than a technology. In case of AJAX it is the browser that initiates the request, but what if the server needs to inform the browser of something ?

Suppose there are 6 clients (browsers) connected to a server viewing the same financial record, if one of the clients updates the record then how do the other connected clients get informed that the record has been updated ? this is a typical problem that a reverse AJAX solution can be used to target.

There are generally 3 methods that can be used for implementation of Reverse AJAX, the details can be read on the original link that I posted earlier

1) COMET or SLOW LOAD

• COMET was coined by Alex Russell , its a technique to keep keep alive HTTP connections

• Bryce Nesbitt's example on slow loading, it uses an frames to keep the connection open

2) Polling

• the browser polls the server after some specified intervals to check if something new has happened

3) PiggyBack

• piggy backing means that whenever a browser requests an operation, you send back the notification with the result of that request, hence piggy backing on the response

I have not used any of the above mentioned techniques but in my opinion a small framework can be built that sends an asynchronous call to the server and listens for the response, the server would only resonse if some event has occured. This would be similar to slow load technique but at least you wont have to use frames with it

I'll try it out if I can get the time, I'm currently preparing for an MCTS exam due in July... so lets see

Indexes in SQL Server

Here's a great article on the ins and outs and do's and dont's of indexing in SQL Server, its a beginner level article but explains pretty much there is to know about indexing.

http://www.odetocode.com/Articles/70.aspx

The above mentioned article focuses on SQL Server 2000, for a comparison with SQL Sever 2005 indexes, click here , on the whole it adds a new type of index for xml datatype that it supports and enhances on previous things, nothing dramatically new.

Links section added

Ive added a new links section to my blog, Most them are those which I visit regularly for updates and news (there are others concerning posts :) ).

My favorites have to be the Architecture resource and patterns and practices links, a really good resource if you are a Microsoft developer, but still good enough on general principles of modeling applications etc. I would really recommend the architecture journal which is available form the architecture resource link, it provides a good insight into whats happening in the architecture world, do's and dont's and opinions of many valued professionals in the field

I'm currently involved with e-commerce development, I came across a really good blog for a company which does research and analysis in e-commerce technology. Its a good blog, providing updates on other news and updates in the development frameworks for e-commerce...

Ive added this to the links section as well, by name of next generation shopping...

compilation model differences in .net 1.1 and 2.0

here's a good article on the differences of the compilation models in ASP.NET 1.1 and 2.0, for those wishing to get the debugging database in 2.0 this article also explores that and tells how it can be achieved, debugging database (.PDB) are useful for getting information like like numbers and other information in a stack dump whenever the code breaks

http://odetocode.com/Blogs/scott/archive/2005/11/15/2464.aspx

Object orientation in javascript

In almost ever project I have been involved this is the case with javascript, its only used for validation checks and small time jobs etc by making a js library and writing procedural code.

in most large forms, java script methods keep growing and eventually you cant figure out if they are being actually used somewhere for what purpose or just junk code.

Here’s a link to 3 articles that hope to eliminate the issue by implementing client side functionality in classes similar to object orientation in .NET, and eventually ejecting the JS client framework through ASP.NET server controls

Part 1 : http://www.codeproject.com/aspnet/JsOOP1.asp

Part 2 : http://www.codeproject.com/aspnet/JsOOP2.asp

Part 3 : http://www.codeproject.com/aspnet/JsOOP3.asp

I'd love to get my hands practical on this one!

Modelling business objects with Encapsulated saving and retreival logic

As I mentioned in the last post that I will be discussing modelling of business objects ,
recently i had a chance to implement the idea that had been bobbing in my head for a
while.

Most of the previous projects that I have worked on had a seperate layer for business
data containers, and another layer for converting the business data container into sql
based information. In my opinion this is an unnesescary , it increases readibility and
maintainance issues.

classic business storage model

I wanted to encapsulate this logic of retrieving and saving the business data within the
business object itself. For e.g. I have a customer object name Customer, i could set all
the information in the customer object and call the save method... Customer.Save();

In the same tradition, i also made methods for retreive by id, retrieve by condition
methods as well, retrieve by id populates and returns an object of type customer,
whereas by condition returns a customer collection. (I didnt use the delete operations,
they wernt required in my case)

Encapsulating the save logic within the business object makes it easier to access and use
by developers. The Save logic consists of a single stored procedure that is used both for
updating and inserting new records. the logic to check wehther the current record is to be
inserted or updated if it already exists can easily be programmed in the stored procedure
itself.

Retriev by id simply constructs the object based on getting the record and populating the
business data by primary key

This type of modelling has other advantages as well, For e.g the Order object
encapsulates a cart object and cart items object, when Object.Save() is called, the
Cart.Save() is called which in turn calls the CartItem.Save() is called for each item in the
cart saving them one by one.

Another thing to keep in mind with a broader vision is that not all items are independant,
and you might want to implement transactions. If the items independantly implement their
own save methods this creates a problem.

Addressing this issue requires that all the classes that need to be saved in a transaction
should have a connection and a transaction members to assign them the connection and
the transaction object being used externally. A boolean transactionMode attribute or
property might be helpful as well

proposed business storage model

there are others who have explored this idea, here's a link to peter bells' blog
entryhttp://www.pbell.com/index.cfm/2006/10/20/Objects-or-Services--how-do-you-desi
gn-your-model

this is also the thinking behind CLSA.NET framework
http://en.wikipedia.org/wiki/Component-based_Scalable_Logical_Architecture

do post your comments on these thoughts

Finally free!

At last!, finally after missing the original deadline by quite a large margin, I have managed to complete a small independent system for receiving paypal and google notifications.

currently its undergoing testing, but it has been really exciting to develop it, on recieving the information from paypal / google we can traverse the information and create user accounts and place orders for our system automaticlly, hopefully this will take quite a burden off the operations team.

It was a fun challange to design a generic set of business classes that could satisfy the information needs for our system (we have an ecommerce shopping site) , information returned by google, and information returned by paypal, Although i can still think of a few improvements here and there but for the moment the model will suffice and should be expandable by implementing another checkout system with little modification.

I will post the architecture that I have tried to use for comments from others. Designing a system that is generic enough to be loosely coupled and integrates with the legacy code (by legacy I mean ASP 3 and COM+) has its share of design issues. lets have a look at the model:

Level 1 implementation

Level 2 implementation

There are 2 levels of implementation, the first level implementation requires to deliver the information form our website to paypal and google checkout.

As you can see, the first level implementation consists of an independant .net 2.0 assembly, registered as COM. This assembly consists of all the core business classes and supporting utility classes. This assembly has a set of core classes that accept xml (based on our own schema for information) as input and product respective html button code through a factory method. Along the way while parsing and validating the xml it also populates the business objects that can be used sometime later in the application development lifetime.

The level 2 implementation is concerned with implementing a notification service, that is able to receive notifications from both paypal ipn and google notification API, although a single landing page is possible but it would create a tightly coupled integration and would be difficult to maintain and expand to another checkout provider if it came along.

NotificationListener is a single web app but with 2 entry points, once for the google notification serivice and the other for paypal. These are just simple asp.net 2.0 pages that use the business classes generated earlier in CheckoutProvider.dll.

NotificationListener also implement microsoft enterprise library 3.0, which again i have to say save d a lot of coding effort and debugging, I used it for logging and exception logging. It has a pretty good structure and relatively quite easy to implement

the only place where i couldnt force loose coupling with the legacy application was the data layer, since all the information has to go in the same database as the actual commerce site, I believe that it can be worked out into an independent component with a little effort and its own database. but given the time constraints that wouldnt be possible anytime soon, plus if its written as an independent component there would have to be some sort of middle ware in between to connect it to existing applications.

I'm focusing on designing reusable and maintainable components , I'd love to work on a truly SOA based application but not quite there yet, but this is a start anyway... would love to hear comments about it.

In my next post i'll also discuss an idea that I have implemented in this implementation, about implementing the save/retrieve logic right inside the business entity object... this approach has a number of flaws which I'll comment on later...

Google Checkout vs Paypal Checkout

For the past couple of weeks, I have been working on implementing the XML API for Google and HTML based API for Paypal checkout, it has been an interesting and varied experience.

I personally liked working with google xml API because it seems more extensible and easy to integrate. its a well structured API building on levels of implementation rather than being completely separate implementations, as is the case with Paypal

the documentation is better structured for Paypal, with easy to understand steps.

I found this lacking on the google page, as you sometimes have to search through the developers forum for some specific examples that have not been published on the documentation page

I'll posst some sample code and tutorials here once I'm done with it

Using AJAX

Continuing from the previous post, I'll first post a small tutorial on using ajax note that AJAX is a browser technology, so far I have tested in firefox and IE, ive read somewhere that SAFARI has partial implementation for it, but I didnt check which version was being discussed. first u'll need to create a factory method in your JS file, this method will , depending on the browser, call the correct library and get the ajax object.
it should look something like this:

function GetXmlHttpObject()
{
var objXMLHttp=null
if (window.XMLHttpRequest)
{
objXMLHttp=new XMLHttpRequest()
}
else if (window.ActiveXObject)
{
objXMLHttp=new ActiveXObject("Microsoft.XMLHTTP")
}
return objXMLHttp
}

You can call the function and get the object that will be sending the actual request

var url = “http://www.sample.comm/sample.asp”;
var xmlObj = GetXmlHttpObject(); //get the object from factory method
xmlObj.onreadystatechange = stateChanged;
xmlObj.open("GET",url,false);
xmlObj.send(null);

a line by line explanation of the code

xmlObj.onreadystatechange = stateChanged;

this line assigns the method to an event, whenever the state of the object changes the method you assign here will be called

xmlObj.open("GET",url,false);

here you are setting the options for opening the connection

arg1 = this can be either “GET” or “POST”, “POST” should be used whenever sending data more than 512kb
arg2 = the url of the server page to hit
arg3 = this parameter specifies whether the call should be sync or async

there are two additional arguments that can be passed, I havnt used them but I think they can be used for HTTP basic authentication

arg4 = username
arg5 = password


xmlObj.send(null);

This is the actual call, note that instead of passing null, you can send xml data in the argument.

Now, whenever the state of the object changes the method that you assigned will be called,

xmlObj.onreadystatechange = stateChanged;

lets look at how we can use this event

function stateChanged(){
if (xmlObj.readyState==4 xmlObj.readyState=="complete") {
alert("success");
}
else{
alert("error");
}
}

The readyState can be any one of the following:

0 = uninitialized
1 = loading
2 = loaded
3 = interactive
4 = complete

In this example I have only used the completed state, note that it might be returning its string counterparts depending on the browser

There are other properties which can be used for testing the state of the xml call object
These are:

responseText : get the response as a stirng
responseXML : get the XML DOM object which can then be parsed

status : status number for http such as 200 ok status or 404 error
statusText : get the status as string like “Not Found” or “OK”

from the server side, you can write anything from the page that you have specified in the callback url in Response.Write(“”) , which will become available in the responseText property of the xml obj once the callis complete

similiary , whole forms can be called and retrieved from response.Text, but its not generally a good idea as discussed in the previous post

head over to w3school.com for more tutorials on calling ajax., but this is pretty much there is to it

Parent child forms in ASP

A month ago, I had to modify an ASP form with some functionality.

this functionality existed in another form, so I decided instead of developing or copy pasting the functionality , i would just call that form as a child in a DIV of the parent form using AJAX.

boy was that a bad decision! believe me it works, the whole form can be called in a div using ajax response.

but i ran into a weird problem that the javascript coming in the AJAX call from the child page didnt run, no matter what. even though the js was there in the AJAX response, I could see it in the debugger

I had to write whole of form validation scripting on the parent page, the elements in the child form were available by using

document.getElementById

in js. I warned another of my colleague who got a similar assignment , he wanted to do it with IFRAME element, and get calls at runtime by changing the form action for the form in the IFRAME element

eventually he ended with similar problems that I was having.

there was another problem with browser caching when using AJAX, when the user updates some values in the child form and submits it and refreshes the parent form, although the changes were reflected in database but the browser (specifically internet explorer) kept showing old values which I believe it keeps picking up from the cache.

theres a browser setting that can be set to check for the new instance of the page on every request, but that didnt seem to work either. We deployed the application with this so called "bug".

I'll post some examples related to this issue later on

Using COM interop services

The past few days have been quite interesting.

Recently I've had a chance to work with XSL and XSLT, the experience was just amazing considering the simplicity and ease of use, i'll probably publish a post on using those later...

The microsoft developers community is slowly migrating towards newer and newer development technologies, but there still a lot of legacy applications out there which require some new development. Here is where COM interop. comes in , it provides the ease of development in .NET while still using the class library as COM in legacy enterprise services such as ASP 3.0

I had a chance to work on a component which was developed in c# .net 2.0 as a class library and called in ASP 3.0

using COM interop is pretty straight forward

when developing a class library for COM interop in .NET its important to keep in mind that all classes and methods to be accessed from legacy app should be public.

in .net 2.0 and VS 2005 the methods and classes are not visible to COM by default,

one way is to make the whole assembly accessible to COM , goto the project properties, under the application tab click assembly information and check the make com visible checkbox near the bottom



the other method involves using System.Runtime.InteropServices library and declaring [ComVisible(true)] attributes at class and method level for each of the method and class


Registering the Assembly as COM :

after compiling the assembly needs to be registered as COM using the command regasm.exe

a typical example of what I used was something like : regasm /codebase / tlb

the /codebase switch sets the code base in the registry and the /tlb creates a type library and registers it as well

there is another option available in visual studio 2005 in project properties build tab, theres a checkbox option to register for com interoperatibility, but i havnt personally tried using that option.

after registering the component it should be ready for use. I will consider usage from an ASP 3.0 application as an example

from ASP 3.0, the instance can be created by using

Server.CreateObject("namespace.classname") command

note that you have to provide the namespace and class name to create the object, the namespace is not required if creating the component in VB 6, instead you can call the class using the assemblyname.classname insted

after creating the object you should be able to call any of the exposed methods for the class. As for the arguments, I passed and returned the primary data types (including string) from my methods in object


Weird Errors :

Tracing COM errors is a difficult job, I suggest including some built in mechanism for logging events in the class library so it becomes easier to trace if the object breaks.

another problem I faced during development was non-relevant error messages thrown by ASP 3.0 this really gave me a headache. I spent almost the whole day trying to figure it out when calling one of my methods i kept getting an error that the instance was not set for my object.

The weird thing was that if I wrote Response.Write with the object name, it showed that the instance was created by printing the namespace and name of the class. At first I couldnt really understand that if the instance is created, but as soon as i call a method i got the error that the object wasnt set to an instance

Eventually i figured out that this error was caused when my COM object was breaking when i called the method and it gave this weird error about instance not being set.

Permissions :

we had to deploy on win3k server, I faced a number of issues there regarding security

Inn my class library i had to access registry. while testing the classes in .net test application it worked fine but asp 3.0 pages failed to compile giving the error that access was denied to registry keys.

this problem occurs because by default in IIS 6.0 IUSR_ account does not have enough rights to access the registry, IUSR_is the machine account under which ASP 3.0 apps run

this can be resolved by giving administrative rights to the IUSR account on the machine, but this solution is not feasible for a lot of production environments.

the alternative is to register the .net COM with component services, open the run window type DCOMCNFG in win3k which will open the component services snap-in
follow these steps:

1. add a new COM+ application
2. create a blank server application
3. provide the identity to be used by this com which has enough access rights (such as accessing the registry in my case)

the new COM+ app should have been created now add you component to this application by adding new component, and selecting from the library thats already registered you should see your namespace.classname in the registered components list (you would have to register using regasm.exe before this activity), select the component and add.

I encountered a problem here that one of the classes i created later was not appearing in the registered list. I traced the problem back to visual studio, it seems that even though make COM visible option was selected, the class still wasnt visible.

I had to manually include include the attribute [ComVisible(true)] in the class, recompile it and re-register using regasm (although first you would unregister the previously registered dll using regasm /unregister)

this solved my probem with having not enough access rights to the registry, becuase now i can specify the COM+ application under which identity to run


Win3k Error : could not create object, access denied :

you might get this error while trying to create your com object, to resolve this goto the COM+ application you created in component services, right click and go to the properties window. Under the Security tab there should be an option to enforce security checks/access, uncheck that option



it should now work.


happy COM interoperting!