Thursday, February 19, 2009

Wipro Campus recruitment

I heard this morning that Wipro is cutting salaries any where between 30-40% across the board to adjust itself to global recession.  I also heard from friends in Wipro, the company is also laying off. And this news came as a total surprise that Wipro is continuing to recruit at College campuses. Not sure, how the offers are being given, Wipro is known to postpone joining dates until 'conditions' improve.

Wipro will hold an exclusive campus drive for B.Sc. final year students at Jagarlamudi Kuppuswamy Chowdhary College in Guntur on Sunday. For details, contact Y. Suresh Babu on phone No. ...

Source: Campus recruitment

Saturday, February 14, 2009

Shifting Away From Tables To Entity Models

RDBMS relational data management systems, simply put, most Database systems allow us to create data models of businesses in terms of rows and columns in a table. Tables can be 'related' to indicate dependencies of data between tables. RDBMS was on the best things that ever happened to software abstractions. Every business domain has been modeled to be a set of tables and relations between them.

However, recently another paradigm is taking shape (at least, I only noticed it in the last year), called as Domain Driven Development, DDD for short, that emphasizes the importance of modeling a business in the domain of the business itself rather than dictated by abstractions tied to tables. DDD clout calls them "Entities" (not Tables).  In certain ways, I read this shift from RDBMS to Entity model as "shift from horizontal (table) abstractions to hierarchical (entity) abstractions".

While Entity models may provide a better modeling, in a well designed database model, based on my experience, you may not find any obvious issues with current table models. And definitely no need to remodel your systems to embrace entity models.

Industry Support

There may be lots of players in the Industry that are supporting Entity frameworks, but based on my current and recent work, two companies I must mention here.

Microsoft released ADO.NET based Entity framework and even made it easier to create those entity models from existing RDBMS tables. And almost every .NET project I have looked at recently is buzzing about DDD, ACL (Anti corruption layer) and Entity framework. LINQ is something can not be omitted, of course.

And Google App Engine doesn't support RDBMS abstraction but only support Entity models. (No JOIN support will force you to think different and prefer one over the other, whether you like it or not).

Update 02/19/2009: Google App Engine team explains the scalability reasons behind moving away from RDBMS and creating its own BigTable.

Back to the Future for Data Storage

Building a massive, distributed datastore which can service requests at an extremely high throughput is something that we've focused on at Google. We created something called Bigtable that underlies the datastore in App Engine. The design for Bigtable focused on scalability across a distributed system so it may operate a bit differently than databases you've worked with before, such as not supporting joins. This isn't an accident -- when you build a system that can scale to the size that Bigtable can there's no way to do a general purpose join on data sets that size and still have them be performant.

So does this mean the end of RDBMS abstractions?

As I said earlier, I don't see any reason to remodel existing systems. The abstraction in Entity models could still be carried out with existing Table models. And I am pretty sure RDBMS systems will stay for a longer time, as almost all data is currently stored in some form of RDBMS system. But abstractions for sure, may change.

Google AppEngine : Lazy Data Migration with Versions

If you have already used Google AppEngine to develop an application, you would have already scratched your head around data migration. And Google App Engine is not that great when it comes to Agile style iterations of web development. In its own way, it forces you to design models up front instead of making life easier evolving over time.

I have been developing a community application, 'Yes to Politics' for friends in Andhra Pradesh, India to interact with politics in some strangely different way. More about this app later, let me share about data migration.

I have around 28000 entities in a model and trust me, I tried to cover all my needs in the design of the model up front.  Well, software development doesn't work that way. I realized I needed another property in the model and I needed to give some default value too.

When you add a new property to an existing model, Google App engine doesn't fill the default value for the existing set of entities. So when you access that new property on existing rows, you will meet with exceptions as they never existed for them. You can check if that property exist before accessing it. Well, Python provide no way to do it. The only way to find whether that property exists or not is by accessing it and catching an exception when it doesn't exist. Not a cool way, but that's almost the best method we got.

And, you can not use that new property in queries too that need to search in the existing set of entities. There is a way we can fix it.

Define your new property and then loop through all your entities and set this new default value.  This is no easy job to do on Google App engine. First, you need to setup a new URL and a HTTP handler to take care of this maintenance. Second, you can only update so many entities in one go without exceeding restrictions in terms of time and processing power. So you need to split the task in to easier bits, say, update 10 entities at a time and create a handler to auto refresh every few seconds to take care of all updates. And then run that handler in the browser and wait till it is finished.

I have about 28000 entities and that would mean I have to call that handler almost 2800 times (10 entities at once) and better give some wait between calls to make App Engine restrictions happy. For my model, 5 sec between each call worked fine. Any quicker, App Engine throws an exception. And it took about 25 minutes for me to finish the process.

I thought it will be anyway one time task so didn't regret waiting that long. But then after all that is done and happily using it for a couple of days,  I found that I had to add yet another property.

This time, instead of doing that hard way, I have decided to do something different. Instead of adding the new property I was thinking of, I added a simple integer that will act as a version number for the entity. I followed the same as above and waited another half hour to get it updated. And for the actual property that I wanted to add, I added the new property but deferred updating the property with a default value until that entity is being used. The lazy way. So I just need to update my queries with this new version number logic but don't have to really update all my data at once.

I realized later it was an excellent move. As not all my entities really need that new property added. Whichever entity needs it, will get it when it is accessed for the first time. Your data migration now becomes highly scalable. Now all my models, I begin with a version number, so that I never have to worry so much about data migration when I decide to make changes to my models.

This is not without a downside. We have an additional property in the model and a little overhead of a version number comparison every time entity is accessed. If you are continuously adding properties that may not be required for all entities, then the storage space you save could easily outweigh this new additional property.

But, you decide whether the flexibility of scalable data migration it provides is worth the weight and hassle.

Wordpress Automatic Upgrade : WoW!

Upgrading Wordpress installation is now just a breeze. Just one click away. I logged in this morning to the dashboard and it alerted me that a new version is available and gave a nice sweet button to automatically upgrade. It just took 5 seconds. No downloads, absolutely. I love it.

I use a shared hosting service, and normally it takes a week before they make a new version of the software available so that I can use their one-click upgrade. Now no more waiting. Just click a button. It is easier than anything else you do on Internet. How cool!

Thank you Wordpress team.

Sunday, February 8, 2009

Political Perspectives : Interview with Arvind

Nothing is black and white in Politics. Especially, Indian politics. So do our perceptions about Politics. We don't hate politics. We don't like it either. Then what does Politics really mean to each of us?

[caption id="" align="alignleft" width="159" caption="Yes To Politics"]Yes To Politics[/caption]I set out to find myself an answer on what it really mean and how it would impact. And I would like to know what do you all think about it. And understand your take on Politics. I would like to call this series of exploration, "Political Perspectives".

And I am happy to share the first of its kind, an interview on Politics. I asked one of my friends, Arvind, a few questions to begin this series. Thank you Arvind for volunteering and sharing your perspective on Politics and thanks a ton for allowing me to share your views to the 'Yes to Politics' community.

Read the transcript of the interview here.

Tuesday, February 3, 2009

Bidding adieu to 37 signals' tadalist for google tasks

With google gmail tasks now available on iPhone, it's time to bid adieu to 37signals' tadalist.

Gmail tasks app is little better than tadalist. But the real reason for the switch is tadalist is a silo app, where as tasks is integrated into gmail and so you can access it with single sign on along with whole bunch of other apps I am already using.

With this, I have replaced yet another silo app in preference to an app from google family.