Database Diver's Diary

Learning Hadoop

These days I have started learn Hadoop at last. I planed to learn this promising technology for a while and finally find a time to do it. Currently, I'm learning the Hadoop architecture and what functionality it can provide that an RDBMS can't do.

As a part of learning the product, I downloaded two VMs (VMWare) with preinstalled single-cluster Hadoop installations. One VM is available from Apache itself. It is running Ubuntu, which I'm not so accustomed to it, it is very minimalistic and doesn't provide any GUI. The other free VM is from Cloudera, it can be downloaded here. It is based on CentOS 5.8 and provides more features and additional products installed than the VM from Apache.

I just want to point out to a caveat that I hit with the Cloudera VM. By default, the VM allocates only 1 GB RAM. To my surprise, any attempt to run even the simplest WordCount example resulted in "out of memory" errors from Java. Under the default RAM allocation, the VM can become virtually inaccesible very easily because there are a lot of Hadoop-related running on it. Fortunately, the workaround is quite simple. Allocate at least 2GB RAM to the VM and everything will work much better. Most probably, there are additional solutions for "out of memory" problem, related to tuning of "-Xmx" parameter, but just providing slightly more RAM to the VM is much simpler. I hope that this tip will save time to another Hadoop beginner like me.

-Xmx

ASE 15.7 ESD#2 - not what you probably thought

Sybase has released ASE 15.7 ESD#2 a number of weeks ago. Not so important event at the first look. Now, I'm in the process of upgrading one of my OEM customers to ASE 15.7 and decided to check what kind of bugs have been fixed in the last EBF. To my great surprise, it appears that Sybase introduced tons of new features in the EBF! I can only guess why all these new features were not even mentioned in 15.7 GA. Some of the new features (not in the order of importance):

• REORG REBUILD TABLE doesn't block and can be performed online. Previously, only rebuilds of indexes could be performed online. May be useful for fixing of fragmentation in some cases.
  
  
• MERGE and SPLIT of semantic partitions is now supported. As far as I understand, EXCHANGE of partitions is still not available. I'm not sure if Sybase is going to implement this in the future, this would be very useful.
  
  
• Improvements in parallel index creation. I have not tested it yet. May help to reduce maintenance or application upgrade downtime in some cases.
  
  
• At last, ASE supports materialized views. For some reason, Sybase calls it "Precomputed Result Sets". In my opinion, it is the right step towards more proper support of mixed-load applications, not just pure OLTP.
  
  
• Hash-based UPDATE STATISTICS (without sort). Sounds promising. Also, UPDATE STATISTICS now generates progress messages.

See the full list of new features here.