Viewing current PostgreSQL Queries

I am a heavy MySQL user, so I constantly use a SHOW PROCESSLIST; to view how my developers or software are using the test database server. So now that we have more than one project using postgresql and since locking down the cpu is becoming more and more common I had to figure out a way to view the queries that are being executed on PostgreSQL.

I found this:

SELECT * FROM pg_stat_activity;

but since PostgreSQL turns off the storing of query strings all I could see was who where using the databse. So in the postgresql.conf file (in FreeBSD that's in /usr/local/pgsql/data/postgresql.conf) i just uncommented and edited the line with this option:

stats_command_string = true

Then just restart the postgresql and it is ready!

Converting numerical bases in python

Working with hexadecimal, binaries, and other bases is becoming common at work, at first we try to use custom functions to convert from and to some bases, especially hexadecimal. But other languages have built-in methods to do that conversion, and so python too.

First to convert and hexadecimal string to an integer we use:

int_number = int('ab1', 16)

The variable int_number will be 2737. That is ab1 hex to integer conversion. Now what about the other way, integer to hexadecimal, that would be:

hex(int_number)

And that would return '0xab1'. Note that it returns a string and has the 0x notation, we could use that notation also in the first conversion like this:

int_number = int('0xab1', 16)

So as you can see we pass a parameter to the int function to indicate the base, if we dont pass any python assumes that we are using a 10 base. As you have also seen there is a hex built-in function but there is no binary, or any other base built-in, so since we work a lot with binary we built a small function to do the work. There are hundreds of ways to do this, using recursion, whiles, fors, lamdas, etc. We end up using this function:

def dectobin(number):
if number < 1:
return ""
else:
return dectobin(number / 2) + str(number & 1)

Hope this works for you.

FreeBSD 7.0 Beta1 Available!

Since I first tried FreeBSD on the server side I have a preference for it over Linux. I think it has superior performance, not that I have done any real benchmark but I have, sort of, felt it. Anyway some friends have done some tests in the past, but tests or not I still think it is a rock solid OS. My experience comes from FreeBSD 4.11 and 5.3, thos two versions were probably the best, now that they are both out of maintenance i use a mix between 5.5 and 6.0. But for some reason those two versions have not been as solid as the 5.3 or 4.11 were. In most cases it was a hardware compatibility problem.

Now FreeBSD 7.0 is nearing its release, probably in December we will have a brand new FreeBSD. I expect to see better hardware support and much more stability in general. So in order to accomplish that I hope to test the new beta release and any other change in the future. Just hope to have some time at work.

Komodo Edit

As a developer I have been searching an editor or IDE that could adapt to every need I could have. This search has been frustrating and not productive. Finally I have accepted the fact that there is no tool that will accomplish what I expected so now I work with a combination of utilities and editors.

Normally my developer time is spend in Linux, there I use gVim or vim. Sometimes when I have to work from home or a client I do it on my laptop that runs Windows Vista. There I recently have been using Komodo Edit. In the search for the perfect IDE I have seen ActiveState Komodo IDE tons of times, but it is available freely only as a Demo. Now ActiveState has released Komodo Edit completely for free and it comes with a set of features that covers my needs.

Also Komodo Edit can be extended using xpi files (the same as Mozilla Firefox and Thunderbird). And there is a community site where you can find this extensions. There are few extensions available now, but when the userbase grows I'm sure they will grow in number.

The features I use more or love more are autocomplete, the calltips, the Django syntax highlighting, block ident/unident, block comment/uncomment, and the multi-edit capability. Even though the space in the laptop screen is reduced (13" compared to the 19" + 19" + 17" that I use at my desktop), I can navigate with very little effort through all my code.

There is even a vim emulation mode! I haven't tried it yet, but for heavy vi users it could be a choice if you wish to explore other editors. There is a linux version but for now I still prefer to use only gVim. It is enough.

So the search is over for now, maybe in a couple of months I will go back to the search of the perfect tool, but for now I think I can use this tools and be productive enough.

Django Date Added and Modified

There has been some discussions in the django-developers list about removing auto_now and auto_now_add helpers for the date related fields. Since this functionality is very useful in those discussions some methods has been suggested, but i think that there has been no further advance in this subject.

For a new project I have been working on we are using trunk code and we want to be sure that for the launch date we still have full functionality, so we created 2 custom fields in order to be able to use them directly on the models and have the desired values automatically set. They work for DateTimeFields only, but the idea could be used also for DateFields.

In a file named fields.py we put this:

from django.db import models
import datetime

class AddedDateTimeField(models.DateTimeField):
    def get_internal_type(self):
        return models.DateTimeField.__name__
    def pre_save(self, model_instance, add):
        if model_instance.id is None:
            return datetime.datetime.now()
        else:
            return getattr(model_instance, self.attname)
           

class ModifiedDateTimeField(models.DateTimeField):
    def get_internal_type(self):
        return models.DateTimeField.__name__
    def pre_save(self, model_instance, add):
        return datetime.datetime.now()

Then in any models file we can import them and just use! Something like this for example:

class Location(models.Model):
    """ Some place's ubication """
    name = models.CharField(_('Name'), maxlength=64, blank=False)
    date_added = AddedDateTimeField(_('Date added'), editable=False)
    date_modified = ModifiedDateTimeField(_('Date modified'), editable=False)
    

I hope the community finds this useful!

Search and Replace in Vi

We are working at a client adapting an already developed project so they can do their work with it. The project was developed using django, first with 0.91, then with 0.95, now we are going to use svn trunk. So one of the things that we are using, and a lot, is just plain search and replace. I am currently developing a lot on my Vaio Laptop, using Windows Vista as OS, and Komodo Edit as editor. It has great Find and Replace capabilities, well really i don't know if that could be complicated.

So, the server is a Centos 4, some problemas with the drivers didn't allow us to use our classic FreeBSD, we are still dealing with the issues of using python 2.3 instead of 2.4 which we are used to. So that and the trunk thing are the main reasons to do a lot of searching and replacing.

A great deal of deveolpment is done at a workstation, there it was easy to follow the GUI and just point and click. In Centos it is another story, I use vi or vim normally, even at my workstation at Aureal I use gvim and I love it. But the thing I never learned was to search and replace using plain vi commands. To make things worse I have, like, 2 or 3 points of experience out of 100 in sed. So I knew that it had to be a similar syntax, the classic:

:s/oldstring/newstring/g

So I tried that, but no, it worked only to replace the oldstring in the current line, that was not what I wanted. So once again google knows it all. It gave me the simple and expected answer in the first returned link. It confirmed that the above string just replaces oldstrings in the current line. But, as vi is a great great editor, you can give the command more parameters, so I learned that you could pass a range of line numbers to apply the search and replace, so I just thought that I could use something like this:

:1,$s/oldstring/newstring/g

On a side note, I just recently learned that the $ means "last line". That really helps a lot! But it turns out that there is a shorter way to write this using:

:%s/oldstring/newstring/g

So the % would be equivalent to "1,$".

And thats it, I just learned that this week. And it is useful! I know, for a developer with 3 years of experience not knowing that would be like a sin, but I just never need it like a did today.