Do you sometimes feel like you missed something completely obvious, that you should really know about, that could have helped you many times, but you just had no idea that it exists? Something, that you simply assumed “it’s not here, move along”? I felt this way today. How did it happen? Well…
Author Archives: Michał
Developer’s Easter Eggs
Did you ever forget posting a blog post that you’ve prepared earlier, the other day? So did I! Some time ago I prepared a short post for April Fools’ day, containing two pieces of code that made me smile recently. However, better late than never – I publish it today! First one is from Cassandra source code and the second one is from a project that company I work for is working on (hopefully it’s not violating my NDA! ;-) ).
Cassandra, HintedHandOffManager class – inspired by (probably fake) quote from Bill Gates about the amount of memory people may need for their computers:
// 64k nodes ought to be enough for anybody. private static final int LARGE_NUMBER = 65536; |
And the second one:
year = date[:4] # This is going to fail when we reach year 10000. Not my problem. |
Warning! It will probably fail to use these quotes as a joke on a party! ;-)
Difference between Cassandra’s Consistency Level ANY and ONE
Just a short note this time. I’ve noticed that there’s a thing that some of the people miss when trying to understand Cassandra’s data model and the two “weakest” Consistency Levels for writes: ANY and ONE. What’s the difference? For CL.ONE it’s required that at least one replica responssible for a specific key, pointed by Partitioner, is able to receive a request and write the data. For CL.ANY it’s required that request is received by any node, including the node that is not responssible for a key. In this case Hinted Handoff will be used and data will by send to proper replicas when they’re available. However, keep in mind that it may happen, that a node that received such query will crash before any of the replica nodes will be able to receive a HH request. In this case, if you fail to recover that node (i.e. because of hardware failure) these data will be lost forever. You have been warned.
Running Flask app with gunicorn using a Flask-Script command (on Heroku)
About two weeks ago I felt a huge need of learning something new – a language, framework… Anything! Just to make my brain work a bit harder. After looking around for a few days (sorry Scala and Erlang, you have to wait a bit longer!) I decided to become more familiar with modern cloud application platforms which are becoming more popular these days (or Paas model in general). Because I think that real projects are much better that rewriting tutorials and reading docs, I decided to write a small Flask web app and deploy it on Heroku. After a few days of learning Flask, coding and running my app on a built-in Flask server I decided to move to a bit more production-ready stage by running it using Gunicorn. And here the story begins…
Removing Cassandra snapshots from all the nodes at once
Sometimest you might want to clean your nodes by removing some snapshots that you don’t need. Even if you did not create them, they might be there – Cassandra makes them before scrub or before truncate. However, removing them one by one from the whole cluster might be a pain, so I wrote a short script that does it.
MX4J – simple web interface for Cassandra JMX
Just a quick tip if you don’t like JConsole or you just prefer to have web UI for Cassandra’s JMX – there’s a quite ugly but usefull tool called MX4J and here’s how to make it work with Cassandra.
Safe Cassandra shutdown and restart
If you are a Cassandra user you’re probably experienced enough to know how to stop or restart Linux services – that’s an obvious thing. However in some cases it might be a problem if a service you turned off goes down, especially if other services have been using it. While Cassandra is very robust and crash-safe (pkill -9 cassandra works fine ;-) ), it’s never a bad idea to do things in a way that minimizes the risk of something going wrong. The other advantage of clean Cassandra restart procedure is saving some startup time. Here is how to do it.
Make SimpleAuthority work with Cassandra 1.1.6 permissions
A few days ago I was about to upgrade Cassandra cluster from 1.1.0 (+ Authentication patch I wrote) to 1.1.6, but – a bit surprisingly – I realized that something is wrong with new Cassandra. I had no problems with creating keyspaces before, as I set proper modify-keyspaces property in access.properties, but after the upgrade it stopped to work. After a short investigation I found out that there were some significant changes in Cassandra’s Permission system which broke SimpleAuthenticator. This article is about how to make it work again.
I am alive!
Just for people who care – I’m alive and I’m fine. I had a lot of work recently, including wedding preparation, but this blog is not dead! I’m still coding, I’m still hacking and I will be posting again soon! Stay tuned! :-)
VirtualBox multi-server development configuration
It’s not always enough to have only one virtual machine for testing purposes. More – it’s not enough in most of the cases. One of such cases is – for example – when you want to try a multi-node configuration for software like Hadoop or Cassandra, or make a failover test of your system. For me most comfortable and useful is to have a configuration that allows guest system to use the Internet (usually it’s a default, NAT mode) but also makes it possible to easily connect to guest system from host system (not by port forwarding) and makes guest systems able to “talk” with each other – it’s good enough to mimic most of production configurations I use. To do this we need to setup a two-network-cards configuration for alle the guest systems we have. This article is about how to make it work.