My SQL2005 Diary – Part1

[ad]

At the place I pretend to work, the time has come that most developers equally fear and love, upgrade time. We’ve been using MSSQL2000 for 90% of our work for about 4 years now, and it’s served us well, but when a change as big as 2005 server comes along, you have to make the leap and upgrade. I suppose a little background is in order, but I’ll have to keep it fairly general as we have some strict rules on what we talk about with people outside the development team.

What we do now

The company I work for is a travel company, one of the big ones, and as with most big travel companies we do a huge variety of things. We own resorts, broker our own insurance, sell for third parties, sell our own holidays, own/rent cruise ships, provide resort management for small hotels, and many other things, all of which is managed through 3 internal sites. We handle the telephone auto-diallers in the call centre, stock-management at our red-sea resort, the links to the main UK flight database, the payment system, our SMS marketing servers, basically, everything.
We have 3 main centres, our corporate headquarters in America, the headquarters in the UK and 1 huge sales centre in the UK also. In addition to that we have either fixed line or internet linked terminals at all our resorts, most of the major airports, all of which connects to our headquarters in the UK(It’s an ex-cupboard upstairs). Because of the international nature of our business, and the resort links the sites must run with 100% uptime 24/7, even though they are all internal.

The sites run on a variety of different platforms, but the vast majority run on old style ASP and SQL server 2000, with a heavy focus on SQL server. To put the workload in perspective, our ASP apps use approximately 5% of our server’s total resources, with SQL server taking the other 95% and another magical 1% running Reporting Services (An excellent application if you’ve never used it). We have a multitude of databases, but we currently run on 4 SQL servers with the databases split as equally as we can get them to avoid having to deal with load balancing. The databases range greatly in size, from a few MB for the HR database, too over 50GB for the lead details database (Call centre data).

Why were upgrading

Due to the size and complexity of the database, performance is extremely important and we have our indexes and maintenance jobs tuned to absolute perfection or the entire thing would come crashing down around us, and we would have a lot of angry people looking to have our heads. But recently we have hit SQL server 2000’s “roof”, which is one of the reasons MSSQL has never challenged Oracle in the big enterprise market, and its proving a big problem for us. SQL server 7 was never meant to be an enterprise level database server, and in typical MS style a lot of SQL server 2000 has come from that original code, as have a lot of the problems, mainly its inability to handle truly massive database. 2005 fixes this.

SQL server 2000 was also limited in that it handled everything via transactions and locking, so if you want to retrieve data from the database in an editable format you have to basically lock that information so nobody else can access it. This can cause all kinds of problems, such as one user being told they can’t perform an action, because their locking themselves (Usually through bad coding) or a deadlock which is data being altered while they are waiting for a lock to end. 2005 borrows from Oracle in that is uses a combination of locking and versioning, which takes a copy of the data, performs the action on it and then puts it back into the database. This presents its own problems, but it does mean users can always get to their data.

There are also some significant coding changes, including some very cool stuff that is new to database servers as a whole. The ability to include code from other languages is one of the main talking points, which basically allows you to execute .net code within your stored procs. This may not sound so great, but you have to consider how it changes the way a DBA will work. At the moment database code needs to be specific, because speed is always an issue the server has to constantly optimize the way it works, and it can’t do this with vague and dynamic code. For example…

Select * from Invoice

Would bring back everything from the invoice table. But what if we just wanted a price field?

Select Invoice.Price From Invoice

That’s easy enough. But what if we wanted the gross price, for example, from insurance items, but the net price for everything else. We would do this(Pseudo-code);

Select (if Invoice.catagory = ‘INSURANCE’ then Invoice.Gross else Invoice.net end if) from Invoice

Again, it looks simple enough, but unfortunately the real code to do this is very complicated and grossly in-efficient at the moment, not to mention completely impossible in certain situations. In 2005 the method above would be perfectly legal, and using Microsoft’s CLR compiler to pre-compile the code, it’s considered adequate (It’s still not as good as plain SQL, but its good enough). This and the performance improvements in the new server would be enough to warrant an upgrade on their own.

What were doing next

We have setup 2 MSDN’d 2005 servers and mirrored our web server as a test bed for upgrading our code. Fortunately the vast majority of our code will still work, but to take advantage of the upgrades and new features we will have to re-write vast swathes of code. And all of our 500+ DTS’s and jobs will have to be completely re-written. And then comes the fun of learning an entirely new interpreter and compiler, and tuning it for maximum performance.

I’ll keep you updated