Tuesday, October 31, 2006

Technology compendium

Try this link. Closer to a technology compendium. Good work, though!

Tuesday, October 24, 2006

Synthesis - The little known and used tool

Today, I came home a little early after a very nice team outing, and was thrilled to see my engineering classmate. We have recollected our good old engineering days, whereabouts of our classmates etc etc. It was nice to know that most of the chaps are doing well and many have preferred Software industry or rather IT as it is called here. And we talked about how many are working as developers, business analysts, release engineers etc. [Btw, programmers, designers, architects all belong to the developer category, my way of grouping ;-)]. The next topic was how many turned out to be good developers (Salary wasn't considered as an yard stick)

One of the things that spurred interest is a subjective equation, in the context of development or assessing a developer, which my friend read in some book; simple but elusive

Good = Acumen * Dexterity

Acumen is the gamut of technologies the person is aware of and dexterity is the extent of deftness exhibited in applying the known/learnt stuff. Acumen can be enhanced through trainings, reading books, learning from colleagues/peers, on the other hand dexterity is something that does not get enhanced through traditional learning approaches. Dexterity cannot be taught and has to be learnt by the individual. The next question would be how to gauge the dexterity.

A simple analogy - any (genuine) mechanical engineer would have studied the subject "Design of Machine Elements". For designing a simple nut and bolt, the right metal has to be choosen, the thread length on the nut has to be calculated, the dimensions have to fixed and the list goes on. To arrive at an "optimal" design, one needs to consider the metallurgical aspects, the shear force and bending moment, production engineering aspects etc. This process of considering all the factors affecting a system is called Synthesis. Essentially, Synthesis is nothing more than looking at all possible angles of the problem and bringing in the knowledge from the related fields and making best use of it.

When designing something, people do some Analysis. Analysis would help in breaking up the problem into small and manageable chunks. This is essential. At the same time, applying Synthesis would add more flexibility, extensibility, and maintainability to the piece being designed. While interviewing candidates, select a person with high synthesis quotient.

Oh boy! I have jotted much stuff. If you feel that this entry is disorganized, put your comments, I'll correct it.

Tuesday, September 12, 2006

Ultra-Scalability

Had been reading an interview with Werner Vogels, CTO of Amazon. It has been very informative. Follow the link Link. I particulary liked the phrase "to be ultra-scalable while maintaining availability and performance".

Thursday, August 03, 2006

Aren't you using Jakarta JMeter

My previous blog entry on "Debugging Webservices" spurred lot of interest and I received mails from many people with lots of questions, pointers, asking help etc.
Many seem to be happy using the SoapUI. Thanks guys!

One of my friends asked me a straight question - aren't you using JMeter to do performance testing of the web services. This was very flat and I wasn't aware that JMeter can be used to load test the WS. There is some tweaking that needs to be done - place mail.jar and activation.jar files in the lib of the JMeter to get the menus and the options displayed correctly. The rest of the setup is similar to any other JMeter configuration. Setup the number of threads, ramp-up period and loop count and hit start. The results will be displayed in a crisp graph.

If you haven't been using JMeter, it is high time you start using it. If you need any help in setting up JMeter WS, send me an email at ravisk@gmail.com.

Sunday, July 16, 2006

Debugging Web Services

Ever wondered how to debug web services?
Debugging web services has been misconstrued by the folks, atleast the people I have been talking to. It is a normal practice to just debug the back end of the WS, example the EJB or an helper class that makes a call to the DB, and claim that the WS is debugged. This, apparantly, doesnot debug the actual service. I decided to come up with a check list for debugging the WS and here it is:

1. The primary stuff, the WSDL should be accessible from the browser
2. Validate request and response bodies against their schema definitions
3. Validate the definitions for WS-I (Web service interoperability) compliance.

The second and third points are of concern. I had been using the TcpMon, that is shipped with Apache Axis so far to see the SOAP messages. This would require "tunneling". Hmmm.. something non-intrusive would be better. I had been looking for some nice tools that does the true debugging of WS. Here is a list of tools and their descriptions:

1. WSUnit - Unit Testing a WS
2. WebService Studio - Invoking the WS, it a .NET program. Given the endpoint, generates the proxies and invokes the methods
3. SOAPUI - A very nice tool with all the required features. It doesn;t limit to just debugging, but provides features to load test and functional test the WS. Really cool!

I installed the SOAPUI and the experience is very good. I strongly recommend it to all folks using WS and wondering how to debug.

Monday, July 10, 2006

CLOB issues in weblogic

I noticed a strange behavior with CLOB in weblogic. The CLOBs would work fine with the main method, but when the same code is run on Weblogic, it throws an "No more data to read from socket" exception. The interesting aspect is, the row in the table gets inserted perfectly including the CLOB column.

The first thing I suspected is with the driver version of the ojdbc.jar shipped with WLS, but that wasn't the problem as the driver I used in the application is same as the one shipped with WLS.

Since, the infrastructure appeared fine, I decided to debug and track the root cause. Debugging the problem brought out some intriguing points. The hack is to create the CLOB column with the default as EMPTY_CLOB() and do a post process after the insertion. The sequence would be:

1. insert some dummy string first cs.setString(4," ");
2. Excecute the proc
3. post process the clob

Clob clob = null;
pstmt = conn.prepareStatement("SELECT clob_col from clob_tbl where id=? FOR UPDATE");
pstmt.setString(1,id);
rs = pstmt.executeQuery();
if(rs.next()) {
clob = rs.getClob(1);
Writer writer = ((OracleThinClob)clob).getCharacterOutputStream();
writer.write(msg);
writer.close();
}
st = conn.prepareStatement("UPDATE clob_tbl set clob_col=? where id=?");
st.setClob(1,clob);
st.setString(2,msgId);
st.executeUpdate();


That is all to be done. Though, there is an additional step in the flow of control, this solution is clean. Whew! some solace.

Tuesday, July 04, 2006

Script Debugger

Well, it has been a while since I coded something on the presentation layer. Infact, I had a very long term association with the Servlets, JSPs, and Struts and sort of bored with the web-db-web routines. The middleware is bit challenging and adventurous. Got some time yesterday, so explored the updates on the presentation layer including JSF. A friend pointed to a script debugger. This was really a cool tool that would literally eliminate the alert() in the javascript code. Progress!
If you'd like to try it out, the link to the tool is
Script Debugger. What more, it is from the Microsoft.

Monday, June 26, 2006

Listening to the PDF

This is nice and cool info from one of my colleagues. If you use Acrobat Reader 7.0, you can listen to the PDF. The following are the shortcuts:
ctrl + shift + b - To listen to the entire document
ctrl + shift + v - To listen to the current page
ctrl + shift + e - To Stop
ctrl + shift + c - To Resume

Very cool, isn't it. Try it out!!

Monday, June 19, 2006

Java Object to Oracle Object mapping

I had a requirement of passing large data to Oracle db and the various options I evaluated are:

1. Using PreparedStatements and batching the updates/inserts: Though this option seems apparent, I didn't consider it as a candidate for the reason that we need to use raw SQL statements and this violates our DB convention to use stored procs and functions. I'm not very sure if this would be fast either. So, dropped this option.

2. Using CLOBs: This would require a lot of change in the existing procs, so dropped it for the sake of reusing the existing procs. Someone, told me that CLOBs do not scale well.

3. Using Arrays: This is an elegant option. Most of the time, the data sent in bulk to DBs is primitive, ie, some numbers or strings or atmost encoded strings (NVARCHAR2). To evaluate sending info using array, I did the following:
a) created a table with just two columns first_name and last_name
b) created a oracle type to be a table of varchar2
create or replace type str_typ_t as table of varchar2(50);
c) created a stored proc to accept the string array as input and insert into the table.
d) The java class required some googling and experimentation, here is the code snippet:

ArrayDescriptor descriptor = ArrayDescriptor.createDescriptor("STR_TYP_T",conn);
ARRAY fnArray = new ARRAY(descriptor,conn,fnames);

That is all to be done. And I'm very happy to note the results. 300 inserts took just over a second!!!. This is Tres cool.

And if you would like to pass a java object to the stored proc and leave the mapping to the driver, the following lines would do the work:

StructDescriptor descriptor = StructDescriptor.createDescriptor("TEST_OBJ_T",conn);
Object[] attributes = {"ABCD","XYZ"};
STRUCT to = new STRUCT(descriptor,conn,attributes);


Some progress today. My PL/SQL skills have been really rusty, need more practice.

Monday, June 05, 2006

Some useful hacks

Well.. it has been a while since I blogged. The primary reason being, I'm very busy with the work and trying to get adjusted to the new project env. The project uses lot of latest technologies, Oracle AQ/Streams, Web Services etc. Some useful hacks I have found last month -

Web Services - This is applicable only for the Axis platform. Axis persists the info of deployed web services to a file by name server-config.wsdd that is created under the WEB-INF. This file, incidentally, has a structure similar to that of the deploy.wsdd file. Instead of creating a deploy file and calling AdminClient from the command prompt, the same entries can be given in the server-config.wsdd and bouncing the server. When the server is started the web services get deployed. This is useful if the deployer of the project is not much aware of the Web services and their deployment processes.
*** Update ***
This is an info I often forget,well, in rush to publish the WS and consume it ;-). If proper description is not given about the methods exposed by the WS, the input arguments are exposed as in0, in1, in2, etc. A few additional lines in the WSDD expose the appropriate and intuitive info about the input parameters:

<operation name="theMethodName" qname="ns:qnameFortheMethod" returnQName="ns:theReturnTypeQNameofTheMethod" returnType="ns:theReturnType">
<parameter name="nameOfTheParameter" type="example-xsd:string"/>
</operation>


I had to recollect this when one of our testing team members said that the WSDL aren't descriptive about the input parameters.

NVarchar2 issue - I observed a strange "No more data to read from socket error" if the return type of the strings is NVarchar2. This was noticed with the Oracle 10g and the latest jdbc driver. Two hacks to overcome this are: a) Change the data type to Varchar2, if you have DB control b) add a java VM argument "-Doracle.jdbc.defaultNChar=true".

Other than work, there seems to be little time for life. Trying best to establish the work-life balance, ;-)

Tuesday, March 21, 2006

Javascript trivia

One of my ex-colleagues was very passionate about Javascript and used to say that Javascript is one of the most misinterpreted and misused language by the developers. I had to recollect this while developing a trivial trim() function. Though there are many ways to implement the trim() functionality, I wanted to do it in a different (um, adventurous way). I explored a bit and found that there is something called "Prototype" for each of the data types used and new functions can be added dynamically. In this case, I wanted to add a trim function, so the following code would add a function called trim to the String data type

String.prototype.trim = function() { }

And now, filling up the function, I used Regular expressions, the leading spaces can be removed by

str=str.replace(/^\s*(.*)/, "$1");

and the trailing spaces by

str=str.replace(/(.*?)\s*$/, "$1");


The whole code would look like

String.prototype.trim = function() {
var str=this;
str=str.replace(/^\s*(.*)/, "$1");
str=str.replace(/(.*?)\s*$/, "$1");
return str;
}

This is may be a trivial discovery but now it has helped us a lot in the Javascipt code, instead of calling trim(str), we can just say str.trim().

Monday, February 20, 2006

JMS backed by Websphere MQ

This is a learning experience about a JMS queue backed by Websphere MQ. My initial understanding was that a message published on the MQ queue wouldn't invoke the onMessage method of a listener/MDB. The notion sounded logical as the Java/JMS equivalent of the message exposed by the MQ JMS is of type com.ibm.mq.JMSBytesMessage and the onMessage expects the argument to be of type javax.jms.Message. Yesterday, we were able to publish a message on MQ and get an MDB listen to the messages. The details follow...

Before getting into the nitty-gritty, some backdrop on the context - our requirement was to process messages put on the MQ "asynchronously". Our knowledge of MQ being naive, the task motivated us and we explored the MQI(java interface for MQ) and played with it by developing some sample code. This led to the roll out of a design using the MQI and the quartz scheduler. I named it the "Pull Approach". A quartz job would poll the MQ at a defined time intervals and retrieve any messages in the MQ queue. Developed the code for the Pull approach and gave a demo of it.

Intrigued by the challenge and exploring more, came up with a second design that would use the inbuilt features of the MQ. A trigger is invoked on publishing a message. The trigger "translates" the message to JMS compatible message and publishes the message on a Topic. This design was baptized as "Push Approach" as the MQ server was pushing the messages. A document has been prepared with both the approaches and with crisp state diagrams made using Rational Rose and circulated it across. There was an inherent flaw in the Push approach, it didn't give implicit support for transactions. There were mixed opinions amongst the folklore about the the approaches with offhsore favoring Pull and onsite for Push.

With adrenalin still gushing, we got more adventurous and explored the JMSAdmin of the MQ. We have been playing around with that for a while but couldnot hit the bush.We decided to give try it afresh from scratch. So, created a context in the JMSAdmin, bound the connection factory and queue to the JNDI, used the same bindings in the JMS configuration on Websphere App server. The prototype was ready for testing. And to our surprise, when we pubslished a message on the MQ, it did invoke the MDB, though there was a ClassCastException for com.ibm.jms.JMSBytesMEssage. The cast exception could be resolved by reading out the bytes out of the JMSBytesMessage and copying them onto the JMS TextMessage.

This was an interesting development, indeed. Although, the earlier two approaches would be promptly decommissioned in favor of the third approach, we are happy that we were able to suggest a robust design that leverages the inbuilt features of the product. The learning curve has taken off. The documentation and the articles found on the IBM developer works site didn't help us much and most of the examples appeared to ratify our initial understanding. Well, being li'l adventurous and thinking outta box does help sometimes. There will a sequel to this blog entry soon with gory technical details.

*** UPDATE ***
There is an article on IBM site that explains in detail about configuring Weblogic JMS for Websphere MQ. Read the article here. I wish the article would have been published earlier that would have saved a few days of my tussle .

Thursday, February 09, 2006

Tussle with Websphere

Has been working with a variety of App. servers like Weblogic, JBoss, and JRun, but never got a chance to work with Websphere. Recently, got a chance to work with Websphere suit of products. First the websphere MQ and then the websphere server itself. The experience with MQ was bit adventurous, as the technology predates JMS. With some trial and error, intuition, googling, got the things up and running. Reb books helped a lot in understanding the internals for MQ.

To test the use cases against our design, we decided to develop a vertical slice of the application. One part of the slice was an MDB that listens to a JMS Queue on the websphere, backed by the MQ. The task is broken to two parts - have an MDB listening to the JMS queue on webspshere and once part 1 is in place, configure a MQ as a backend for the JMS queue. Developing an MDB was no much hassle and it got deployed on the websphere(The server had to be bounced everytime after configuring the Listener Ports and registering the Queue with the JMS server, and after deploying the jar file with MDB, duh, it sucks!). The next step was to publish some messages on the queue and see if the onMessage of the MDB is getting called.

I developed a suit of small utilities to help me in server-side testing, which I call as the fixture. The configuration of the fixture is externalized to facilitate portability. The messaging fixture worked without any errors when it was pointing to Weblogic, JBoss, JRun. But with Websphere, a lot of tweaking of the classpath is required. The following is the summary of the tweaking:

1. First of all, use the JDK that comes with the server, located at ${was_home}/java. If you don't, you'll waste a couple of hours like me.

2. Place these jars in the classpath:
com.ibm.mq.jar
com.ibm.mqjms.jar
dhbcore.jar
ecutils.jar
j2cImpl.jar
j2ee.jar
messagingClient.jar
naming.jar
namingClient.jar
namingserver.jar
webcontainer.jar
Properties folder. Place all or you'll repent.

3. Websphere works on iiop. So the JNDI props will be com.ibm.websphere.naming.WsnInitialContextFactory and iiop://localhost:2809.

To figure out what jars go in the classpath, it took me almost 6 hours. Extensive googling and some experimentation made the things work. It seems most of the people had a similar experience. Looks like I'm going to have some more tussles with websphere in the near future. Why can't the Websphere be as developer friendly as Weblogic.

Tuesday, January 31, 2006

Year.introspect(2005)

Yesterday,I was arranging my book-shelf to accomodate new books and a paper fell-out. It was a receipt of a book purchased from Gangarams book store, Bangalore, dated Jan 31, 2005. The book, Head First Design Patterns, is still my favorite book. After reading the book, there is a conspicuous change in the way I write code. I have undergone a transformation, hehehe. A friend calls it metamorphosis. I have recommended the book to all my colleagues and the next day there were 5 more purchases of the book.

Incidentally, on the same day Jan 31, 2005, I've put my papers at HP. The co-incidence of the events developed a retrospective urge and made me look back and introspect the last year. I had more >5 offers on hand from good companies at that moment. I've truned down all the offers from companies in Bangalore as the real estate prices skyrocketed and the traffic is horrendous. Subsequently, I decided to relocate to Hyderabad, my native place and narrowed down on a medium sized company doing product development for wealth management segment of the financial industry.

The year was of mixed results, sometimes working more than 14 hours a day and sometimes with absolutely no work for weeks together! oh yeah! there was an onsite trip for 3 months to San Francisco. Oflate, I have become a passionate reader[almost a bookworm ;-)] and mostly I'm reading tech books (currently reading Introduction to Algorithms by Thomas Cormen, Geeeee).Things worked out ,more or less,as expected and the year was a break-even, if not lucrative. Wondering what 2006 has in store for me, wink!

Sunday, January 29, 2006

The Jaipur trip

A week long trip to Jaipur had been full of joy and happiness. I was wondering if I could make it to Vivek's wedding amidst so much work. My manager finally relented and approved a leave for 3 days. Weekend + republic day added 3 more days. 6 days of total fun and joy.

I have known Vivek since my engineering days. He is a Telugu speaking Marwadi :-).
The train journey to Jaipur and back was really entertaining. I travelled along with 40 other friends and relatives of Vivek. There was no need to turn on the mp3 player as the kids were playing anthakhasari non-stop. Though, the particpation from elders (including me) was meagre, it was nice to listen to the Hindi songs.

The marriage and the reception were very elegant and full of activity. I never got a chance to see a traditional north Indian marriage, except in the movies. The baraath was the ultimate. We danced and enjoyed the music played by the band. It was all fun and an unforgettable experience. Look at Vivek on the horse..

Wednesday, December 14, 2005

Capital Markets - a Fascinating Journey

I'm very fascinated by the Capital Markets and their workings. Incidentally, my current work is developing a wealth management solution product that shall be implemented for major banks in US. The current task is to develop the Portfolio Performance pages. Business knowledge and understanding of the performance measurement techniques were impediments to roll out the task on time. A business analyst from the client was sent to provide inputs on the domain

With client's presence in the office and questions getting answered instantaniously, we are in a pretty good shape now. This was more of a learning exercise than just coding. Reading the book "Pratical Portfolio Performance Measurement & Attribution" helped a lot in getting upto speed. More specifically, it helped me to get a clear understanding of the Dietz's formula.

I'm delighted on hearing that I was nominated for a week long training on Financial Concepts and Capital Markets, starting next week. More inputs on the Finance Domain. But, I'm still not an investor ;-)

Sunday, December 04, 2005

My New Laptop

I bought a new laptop on the weekend. It is Compaq Presario V4000/4103AP make with ubiquitous disk space and RAM. The most impressive features are the wide screen and the TFT display, playing Age of Empires/Kings on the laptop is a delight. Now, I can do some serious programming on the weekend and play my favorite games - The Commandos and Age of Empires. Has been trying from a long time to learn Ruby, but that remained elusive due to the tight work schedules, client visits. I hope with the laptop, I'll be an Rubyist soon. :-D

Tuesday, November 29, 2005

Humor: Knowledge Theorem

This blog entry is for those cribbing about their low salaries and comparing their salaries with their friends in other companies. Not intended to hurt anyone's feelings and opinions.To be taken in a lighter spirit.

I received a mail today about something called as the Knowledge Theorem, that states "An Engineer cannot make as much money as the Politicians, artists". The author used the following "inductive procedure" to substantiate the theorem.

Two adages everyone is aware of - "Knowledge is Power" and "Time is Money". If represented mathematically, these two adages would be

Knowledge = Power
- eqn. 1
Time = Money - eqn. 2

High school physics has taught us that

Power = Work/Time.

From eqn. 2, Time = Money, so

Power = Work/Money

From eqn. 1, Knowledge = Power, so

Knowledge = Work/Money

Rearranging the quotient and the divisor would result in

Money = Work/Knowledge

The lesser the denominator, the more is the fraction. Hence proved!
If a person is earning less, that means he is more knowledgable. The converse of the statement is, however, not true.

Wednesday, November 16, 2005

The best description of OO

Had been reading an article titled "OO in One Sentence: Keep It DRY, Shy, and Tell The Other Guy" from pragmaticprogrammer. A funny sounding title, but the content explains the essence of OO in a more interesting way. I liked a particular paragraph so much that I'll quote it all in verbatim:
"The best code is very shy. Like a four-year old hiding behind a mother’s skirt, code shouldn’t reveal too much of itself and shouldn’t be too nosy into others affairs. But you might find that your shy code grows up too fast, shedding its demure shyness in favor of wild promiscuity. When code isn’t shy, you’ll get unwanted coupling."

This is the best description of OO I've heard of so far.

Wednesday, November 02, 2005

AJAX Mania

Oflate, the "ajax" is in the limelight. The technology has created so much excitement among the web developer community and eventually is conferred with laurels. Some of the accolades received are "The next big thing", "revolutionary web technology", "web2.0", etc. The technology indeed has a potential and some applications have leveraged and demonstrated it. The most famous and often cited are the google maps, gmail, flickr.

Recently, rediffmail rolled out a new version of their web mail that offers some goodies implemented using ajax - mail preview, instant download of attachments, drag drop unwanted mails to trash, auto suggest of email ids while composing a mail, etc. It is now pleasure to use rediffmail with all the ajax goodies.

We have an ajax implementation in our product to retrieve information at lower levels from a hierarchically organized data, on click of expand/collapse (+/-). The existing functionality retrieved all the data at once, dumped the data behind the div tags and the selected div would be made visible using javascript. This worked fine for smaller amounts of data and for large data, apparently, became a bottleneck in terms of retrieval and rendering times. Subsequently, the issue has been resolved by implementing ajax. When someone suggested the ajax solution to our chief architect, he suggested some concerns and asked us to explore more for an optimal solution:

1. Browser compatibility - there should be a utility to check for the compatibility of the browser for ajax to serve the ajaxified and no-ajaxified versions accordingly.

2. Increase in the server hits for every ajax call

3. Feedback mechanisms during ajax calls and appropriate messages on the screen

4. Processing order of the ajax calls

5. The change should have a least impact on the existing functionality/code and should be maintenable.

These concerns were very realistic and very soon we realized that ajax is not a panacea. The final solution used ajax at only certain points where it added value. My request to fellow developers and friends is not to get carried away by the excitement, understand the true potential of the technology and implement it at a place where it makes sense.