Running Flask app with gunicorn using a Flask-Script command (on Heroku)

About two weeks ago I felt a huge need of learning something new – a language, framework… Anything! Just to make my brain work a bit harder. After looking around for a few days (sorry Scala and Erlang, you have to wait a bit longer!) I decided to become more familiar with modern cloud application platforms which are becoming more popular these days (or Paas model in general). Because I think that real projects are much better that rewriting tutorials and reading docs, I decided to write a small Flask web app and deploy it on Heroku. After a few days of learning Flask, coding and running my app on a built-in Flask server I decided to move to a bit more production-ready stage by running it using Gunicorn. And here the story begins…

Continue reading

Removing Cassandra snapshots from all the nodes at once

Sometimest you might want to clean your nodes by removing some snapshots that you don’t need. Even if you did not create them, they might be there – Cassandra makes them before scrub or before truncate. However, removing them one by one from the whole cluster might be a pain, so I wrote a short script that does it.

Continue reading

VirtualBox multi-server development configuration

It’s not always enough to have only one virtual machine for testing purposes. More – it’s not enough in most of the cases. One of such cases is – for example – when you want to try a multi-node configuration for software like Hadoop or Cassandra, or make a failover test of your system. For me most comfortable and useful is to have a configuration that allows guest system to use the Internet (usually it’s a default, NAT mode) but also makes it possible to easily connect to guest system from host system (not by port forwarding) and makes guest systems able to “talk” with each other – it’s good enough to mimic most of production configurations I use. To do this we need to setup a two-network-cards configuration for alle the guest systems we have. This article is about how to make it work.

Continue reading

Problem with very slow sendmail startup

I was playing a bit with some virtual machines I need for testing, when after a reboot I noticed that sendmail is starting very slow – it took about 3-4 minutes to have it working. I’ve checked the log too see what’s wrong:

[root@hdps01 ~]# tail /var/log/maillog
Jul 11 21:26:43 hdps01 sm-msp-queue[1266]: My unqualified host name (hdps01) unknown; sleeping for retry
Jul 11 21:27:43 hdps01 sm-msp-queue[1266]: unable to qualify my own domain name (hdps01) -- using short name
Jul 11 21:27:43 hdps01 sm-msp-queue[1289]: starting daemon (8.14.4): queueing@01:00:00

So, what was the problem?

Continue reading

Be careful with Hadoop and pbzip2!

Or you may be surprised at one day when you see that your output looks like it’s missing a lot of data. The problem affects Hadoop versions older than 1.4 (according to Jira) and is caused by the misinterpretation of EOS in compressed files, which is interpreted as EOF, so it – obviously – ends reading the file:

https://issues.apache.org/jira/browse/COMPRESS-146

So, if your Hadoop is misbehaving and your output data look odd without any reason – ask your admins if they didn’t change bzip2 to pbzip2.

Combine multiple video frames into one using OpenCV

I’m playing a lot with OpenCV and object recognition recently (I started from Python, but now I’ve moved to C++) and what I wanted to do to make my life easier was putting 4 different video frames (original, thresholded, grayscaled & thresholded and original with marked face). Even if it’s not very difficult task, it took me a while to do it, because – definitely – I’m not an OpenCV expert. This article shows how I did it using the new 2.3 (or – actually – 2.X) OpenCV API.

Continue reading

Very good interactive Git workflow cheat sheet

Few minutes ago I was looking for a Git workflow cheat sheet to verify some rarely-used parts of my knowledge before doing something I might regret. Actually I was hoping to find something very simple (preferrably one pdf page or so), but instead I found this one, which is a very good, interactive webpage. So I decided to share this find with you, because it’s definitely worth it:

http://ndpsoftware.com/git-cheatsheet.html

Git workflow cheat sheet

Git workflow cheat sheet

What’s best in it, it presents everything in a very intuitive, visual way which is easy to understand. If you are looking for a command which will completely revert your commited changes, you can just click on “Local Repository” and see which arrow points to “Workspace” – it’s a git reset --hard one. How about leaving the changes you’ve made, but reverting commit? It’s an arrow with git reset --soft. Brillant!

It’s not only a good thing for people who look for a typical cheat sheet, but also for those who have some problems with understanding git workflow.

I really like it – nice work guys!

More on Cassandra’s SimpleAuthority permissions

Few days ago I had some doubts on how Cassandra’s SimpleAuthenticator and SimpleAuthority really work. I mean – I was not sure of the way I should configure them to get the expected results. It may seem obvious now, but I had to look at source code to find out what is possible and what is not. So, to save your time, here’s a brief description of this.

Continue reading