zliczanie błędów w logach postgresa
Kilka razy mi się przydało więc zapisuję...
for distance in {30..0}; do
date=`date +%Y-%m-%d --date "$distance days ago"`;
echo -n "$date ";
bzcat postgresql-$date*.bz2 |grep -wc ERROR;
done
Wersja ze zliczaniem nie tylko ERROR-ów:
for distance in {20..1}; do
date=`date +%Y-%m-%d --date "$distance days ago"`;
echo -n "$date ";
lzcat postgresql-$date*.lzma \
| perl -nle '$s{$1}++ if /\b(PANIC|FATAL|ERROR|WARNING|NOTICE)\b/; END{for(keys%s){$o.="$_:$s{$_};"};print $o}';
done
column number as column alias in postgres
I have seen such queries many times:
SELECT foo, bar, (some long and complicated SQL subexpression), count(*)
FROM somewhere
GROUP BY foo,bar, (some long and complicated SQL subexpression)
ORDER BY count(*) DESC
This has some drawbacks:
- you have to repeat exactly the same expression two times, which makes query longer
- it is easier to make a mistake while editing the subexpression
Did you know there's a neat shortcut for this in postgres?
SELECT foo, bar, (some long and complicated SQL subexpression), count(*)
FROM somewhere
GROUP BY 1,2,3
ORDER BY 4 DESC
This is known as column number aliases.
I have no idea if this comes from SQL standard or not.
But it saves me quite a lot of typing :)
I know it works at least in postgresql, mysql and informix.
Does it work in your database, too?
Postgres partitioning performance – rules vs triggers
Rafal Pietrak asked a question about postgres performance in partitioning scenarios.
The problem is, in classical partitioning approach you decide into which partition put the data basing only on the inserted data itself.
But we consider also situation when you want to make this decision basing on current database content.
For example we have some "driving" or "routing" table which tells us which partition is currently active.
Please read the above post for more background.
I prepared 4 test cases, for all combinations of rule versus trigger and static versus dynamic aka table-driven partitioning.
Test was performed on PostgreSQL 8.3.3 on Linux, commodity desktop box.
To make things short, here are the results of two test runs (links point to test scripts):
/what is measured: INSERT of 10000 rows/
Partitioning with RULEs, no dynamic routing:
2444.293 ms 2516.314 ms
Partitioning with RULEs, with dynamic routing:
42380.037 ms 39248.666 ms
Partitioning with TRIGGER, no dynamic routing:
14512.787 ms 14669.310 ms (but see the update below)
Partitioning with TRIGGER, with dynamic routing:
13486.808 ms 13904.370 ms
Conclusion:
If you have to do some database lookup to decide which partition data belongs to, use a trigger on master table.
If you have a well defined static set of rules, use PostgreSQL rule system.
Update (2010/06/15)
depesz wrote an article in which he noticed some problems with above test case.
First, he points out that the trigger in "Partitioning with TRIGGER, no dynamic routing" example is horribly written. And he is right. In this particular case, partition assignment logic is so simple it can be expressed in 5 lines instead of 1005 lines of code. This makes trigger-based solution an order of magnitude faster.
Second, he says that bulk inserts are not so common - so measuring them is not-so-relevant. This heavily depends on the flavour of database you are working on. For OLTP systems, depesz's statement is obviously true, but for analytic (OLAP and BI) systems, bulk inserts are more common.
Third, in general I agree that rules should be avoided. Triggers are more readable and understood by most database-literate programmers. Rules ... well the first rule of using PostgreSQL rules is "DO NOT USE RULES".
(end of update)
hstore key rename
hstore is a "Lazy DBA" extension for postgres, which enables storing hash tables in a table column.
When using hstore, sometimes you want to rename hash keys.
#
# create function renamekey( _hstore hstore, _oldname text, _newname text ) returns hstore as
$$ select delete($1, $2) || ($3 => ($1 -> $2)) $$ language sql immutable strict;
#
# \set old_codename '''dizzyflag'''
# \set new_codename '''is_trade'''
#
# UPDATE table_with_hstore
SET flags = renamekey(flags, :old_codename, :new_codename)
WHERE (flags -> :old_codename) is not null;
#
Sortowanie w PostgreSQL
Jarek napisał ciekawy artykuł o sortowaniu napisów w PostgreSQL (i nie tylko, w zasadzie rzecz dotyczy localesów w glibc).
