a:4:{s:5:"child";a:1:{s:0:"";a:1:{s:3:"rss";a:1:{i:0;a:6:{s:4:"data";s:3:" ";s:7:"attribs";a:1:{s:0:"";a:1:{s:7:"version";s:3:"2.0";}}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";s:5:"child";a:1:{s:0:"";a:1:{s:7:"channel";a:1:{i:0;a:6:{s:4:"data";s:217:" ";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";s:5:"child";a:1:{s:0:"";a:6:{s:5:"title";a:1:{i:0;a:5:{s:4:"data";s:12:"Planet MySQL";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:4:"link";a:1:{i:0;a:5:{s:4:"data";s:27:"http://www.planetmysql.org/";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:7:"pubDate";a:1:{i:0;a:5:{s:4:"data";s:31:"Wed, 09 Jun 2010 02:45:01 +0000";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:8:"language";a:1:{i:0;a:5:{s:4:"data";s:2:"en";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:11:"description";a:1:{i:0;a:5:{s:4:"data";s:42:"Planet MySQL - http://www.planetmysql.org/";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:4:"item";a:50:{i:0;a:6:{s:4:"data";s:73:" ";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";s:5:"child";a:3:{s:0:"";a:6:{s:5:"title";a:1:{i:0;a:5:{s:4:"data";s:45:"Making Deletions Fast, by Avoiding Disk Seeks";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:4:"guid";a:1:{i:0;a:5:{s:4:"data";s:26:"http://tokutek.com/?p=1491";s:7:"attribs";a:1:{s:0:"";a:1:{s:11:"isPermaLink";s:5:"false";}}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:4:"link";a:1:{i:0;a:5:{s:4:"data";s:72:"http://tokutek.com/2010/06/making-deletions-fast-by-avoiding-disk-seeks/";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:11:"description";a:1:{i:0;a:5:{s:4:"data";s:4497:" In my last post, I discussed how fractal tree data structures can be up to two orders of magnitude faster on deletions over B-trees. I focused on the deletions where the row entry is known (the storage engine API handler::delete_row), but I did not fully analyze how MySQL delete statements can be fast. In this post, I do. Here I show how one can use TokuDB, a storage engine that uses fractal tree data structures, to make MySQL deletions run fast. Let’s take a step back and analyze the work needed to be done to execute a MySQL delete statement. Suppose we have the table: create table foo ( id auto_increment a int, b int, primary key (id) ) Say we wish to perform the following operation that deletes 100,000 rows: delete from foo where a=1; In MySQL, this statement executes in two steps. First, MySQL finds all the rows where a=1, via a query. The query is equivalent to “select * from foo where a=1;” Then for each row found, MySQL deletes the row by calling handler::delete_row(row). So the time to execute deletions is equivalent to the time to find the rows (T_query) plus the time to delete the found rows (T_change). Or, stated another way: T_delete = T_query + T_change The previous post shows why T_change is faster for fractal tree data structures than for B-trees. Let’s use that fact to see how we can make deletions fast under MySQL. Back to the original problem at hand: delete from foo where a=1; For this statement, T_query is a table scan. Because no index on ‘a’ exists, every element must be processed to find where a=1. For large tables, this can be expensive. The (minor) advantage here is T_change, the cost of removing the 100,000 rows, requires no disk seeks, because rows where a=1 stay in memory as MySQL deletes them. The problem remains, we process too much data in T_query to delete some rows. So how can we speed up a query? Indexing! Or specifically, add an index on ‘a’. The schema is now: create table foo ( id auto_increment a int, b int, primary key (id), key (a) ) Now what is the cost of “delete from foo where a=1;”? Well, for TokuDB, InnoDB, and MyISAM, T_query requires roughly 100,000 disk seeks, because point queries are needed to retrieve the entire row. The key (a) is not a covering index. The (minor) advantage here (again) is that T_change has no additional disk seeks, because the query does the disk seeks necessary to bring the rows where a=1 into memory, both in the primary index (or .MYD file for MyISAM) and secondary index. The problem remains that we are still doing about 100,000 disk seeks! So how can we further speed up T_query? Remember, we want to make the query “select * from foo where a=1;” faster. Clustering indexes! The schema looks like: create table foo ( id auto_increment a int, b int, primary key (id), clustering key (a) ) Suppose MyISAM and InnoDB supported clustering indexes (they don’t). What is their cost for performing the deletion? Well, T_query becomes much faster, because it would be a range query as opposed to 100,000 point queries. But T_change is still expensive. As explained in my last post, B-trees require disk seeks for deletion when the row is not in memory. The rows that need to be deleted in the primary key are not in memory and require disk seeks. So, for MyISAM and InnoDB, we are still stuck with at least 100,000 disk seeks. For this reason, some erroneously think “clustering keys do not help here”. Now what about TokuDB, a storage engine that uses fractal tree data structures? T_query is also fast, because a range query is performed. But T_change is ALSO fast, because the deletions in the primary index do NOT always require disk seeks. This is where the advantage of fractal tree data structures over B-trees comes into play, allowing clustering keys to speed up a deletion procedure. So, the moral of the story is this. Deletions are a combination of queries and value changes. To make deletions fast, both the query and the value changes need to be fast. With B-tree based storage engines, users may be hesitant to add indexes to speed up the query, due to the added cost of value changes in the index. With TokuDB, this tradeoff does not exist, because TokuDB uses fractal tree data structures. Using a clustering key to drive down the cost of T_query, and using fractal tree indexes to keep the cost of T_change down, leads to very fast deletes.";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:7:"pubDate";a:1:{i:0;a:5:{s:4:"data";s:31:"Tue, 08 Jun 2010 16:19:41 +0000";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:8:"category";a:7:{i:0;a:5:{s:4:"data";s:8:"TokuView";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:1;a:5:{s:4:"data";s:6:"B-tree";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:2;a:5:{s:4:"data";s:6:"delete";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:3;a:5:{s:4:"data";s:9:"disk seek";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:4;a:5:{s:4:"data";s:13:"Fractal Trees";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:5;a:5:{s:4:"data";s:5:"mysql";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:6;a:5:{s:4:"data";s:6:"TokuDB";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}s:40:"http://purl.org/rss/1.0/modules/content/";a:1:{s:7:"encoded";a:1:{i:0;a:5:{s:4:"data";s:5061:"

In my last post, I discussed how fractal tree data structures can be up to two orders of magnitude faster on deletions over B-trees. I focused on the deletions where the row entry is known (the storage engine API handler::delete_row), but I did not fully analyze how MySQL delete statements can be fast. In this post, I do. Here I show how one can use TokuDB, a storage engine that uses fractal tree data structures, to make MySQL deletions run fast.

Let’s take a step back and analyze the work needed to be done to execute a MySQL delete statement. Suppose we have the table:

create table foo (
	id auto_increment
	a int,
	b int,
	primary key (id)
)

Say we wish to perform the following operation that deletes 100,000 rows:

delete from foo where a=1;

In MySQL, this statement executes in two steps. First, MySQL finds all the rows where a=1, via a query. The query is equivalent to “select * from foo where a=1;” Then for each row found, MySQL deletes the row by calling handler::delete_row(row). So the time to execute deletions is equivalent to the time to find the rows (T_query) plus the time to delete the found rows (T_change). Or, stated another way:

T_delete = T_query + T_change

The previous post shows why T_change is faster for fractal tree data structures than for B-trees. Let’s use that fact to see how we can make deletions fast under MySQL.

Back to the original problem at hand:

delete from foo where a=1;

For this statement, T_query is a table scan. Because no index on ‘a’ exists, every element must be processed to find where a=1. For large tables, this can be expensive. The (minor) advantage here is T_change, the cost of removing the 100,000 rows, requires no disk seeks, because rows where a=1 stay in memory as MySQL deletes them. The problem remains, we process too much data in T_query to delete some rows.

So how can we speed up a query? Indexing! Or specifically, add an index on ‘a’. The schema is now:

create table foo (
	id auto_increment
	a int,
	b int,
	primary key (id),
	key (a)
)

Now what is the cost of “delete from foo where a=1;”? Well, for TokuDB, InnoDB, and MyISAM, T_query requires roughly 100,000 disk seeks, because point queries are needed to retrieve the entire row. The key (a) is not a covering index. The (minor) advantage here (again) is that T_change has no additional disk seeks, because the query does the disk seeks necessary to bring the rows where a=1 into memory, both in the primary index (or .MYD file for MyISAM) and secondary index.

The problem remains that we are still doing about 100,000 disk seeks!

So how can we further speed up T_query? Remember, we want to make the query “select * from foo where a=1;” faster. Clustering indexes! The schema looks like:

create table foo (
	id auto_increment
	a int,
	b int,
	primary key (id),
	clustering key (a)
)

Suppose MyISAM and InnoDB supported clustering indexes (they don’t). What is their cost for performing the deletion? Well, T_query becomes much faster, because it would be a range query as opposed to 100,000 point queries. But T_change is still expensive. As explained in my last post, B-trees require disk seeks for deletion when the row is not in memory. The rows that need to be deleted in the primary key are not in memory and require disk seeks. So, for MyISAM and InnoDB, we are still stuck with at least 100,000 disk seeks. For this reason, some erroneously think “clustering keys do not help here”.

Now what about TokuDB, a storage engine that uses fractal tree data structures? T_query is also fast, because a range query is performed. But T_change is ALSO fast, because the deletions in the primary index do NOT always require disk seeks. This is where the advantage of fractal tree data structures over B-trees comes into play, allowing clustering keys to speed up a deletion procedure.

So, the moral of the story is this. Deletions are a combination of queries and value changes. To make deletions fast, both the query and the value changes need to be fast. With B-tree based storage engines, users may be hesitant to add indexes to speed up the query, due to the added cost of value changes in the index. With TokuDB, this tradeoff does not exist, because TokuDB uses fractal tree data structures. Using a clustering key to drive down the cost of T_query, and using fractal tree indexes to keep the cost of T_change down, leads to very fast deletes.


PlanetMySQL Voting: Vote UP / Vote DOWN";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}s:32:"http://purl.org/dc/elements/1.1/";a:1:{s:7:"creator";a:1:{i:0;a:5:{s:4:"data";s:13:"Tokuview Blog";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}}}i:1;a:6:{s:4:"data";s:63:" ";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";s:5:"child";a:3:{s:0:"";a:6:{s:5:"title";a:1:{i:0;a:5:{s:4:"data";s:40:"Oracle resources for the MySQL Community";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:4:"guid";a:1:{i:0;a:5:{s:4:"data";s:38:"http://ronaldbradford.com/blog/?p=2870";s:7:"attribs";a:1:{s:0:"";a:1:{s:11:"isPermaLink";s:5:"false";}}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:4:"link";a:1:{i:0;a:5:{s:4:"data";s:83:"http://ronaldbradford.com/blog/oracle-resources-for-the-mysql-community-2010-06-08/";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:11:"description";a:1:{i:0;a:5:{s:4:"data";s:1139:"While I have spent a lot of time recently helping the MySQL community interact with and integrate with various Oracle User Groups including ODTUG, IOUG, NoCOUG, NYOUG, DAOG I thought I’d share some resources for the MySQL Community that wanted to know more about Oracle. The Oracle family of products is huge. You only have to look at the acquisitions via Wikipedia to get an idea. The first thing is to narrow your search, e.g. Database, APEX, Middleware, BI, Hyperion, Financials, development via Java, PHP or Oracle Forms etc. While Oracle is a commercial product you can download all software for FREE via Oracle Technology Network. There is also documentation, forums, blogs and events. Some Oracle bloggers I have already been reading however I’m expanding my list. People you may want to consider include: Cary Millsap,Lewis Cunningham, Debra Lilley, Dimitri Gielis,Duncan Mills, Edward Roske, Mark Rittman, Scott Spendolini, Tim Tow, Tom Kyte If you want a comparison of the Oracle and MySQL community, be sure to also check out Sheeri Cabral’s keynote address at the 2010 MySQL User Conference for reference.";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:7:"pubDate";a:1:{i:0;a:5:{s:4:"data";s:31:"Tue, 08 Jun 2010 14:15:22 +0000";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:8:"category";a:5:{i:0;a:5:{s:4:"data";s:9:"Databases";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:1;a:5:{s:4:"data";s:5:"MySQL";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:2;a:5:{s:4:"data";s:6:"Oracle";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:3;a:5:{s:4:"data";s:12:"Professional";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:4;a:5:{s:4:"data";s:3:"OTN";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}s:40:"http://purl.org/rss/1.0/modules/content/";a:1:{s:7:"encoded";a:1:{i:0;a:5:{s:4:"data";s:2420:"

While I have spent a lot of time recently helping the MySQL community interact with and integrate with various Oracle User Groups including ODTUG, IOUG, NoCOUG, NYOUG, DAOG I thought I’d share some resources for the MySQL Community that wanted to know more about Oracle.

The Oracle family of products is huge. You only have to look at the acquisitions via Wikipedia to get an idea. The first thing is to narrow your search, e.g. Database, APEX, Middleware, BI, Hyperion, Financials, development via Java, PHP or Oracle Forms etc.

While Oracle is a commercial product you can download all software for FREE via Oracle Technology Network. There is also documentation, forums, blogs and events.

Some Oracle bloggers I have already been reading however I’m expanding my list. People you may want to consider include:

Cary Millsap,Lewis Cunningham, Debra Lilley, Dimitri Gielis,Duncan Mills, Edward Roske, Mark Rittman, Scott Spendolini, Tim Tow, Tom Kyte

If you want a comparison of the Oracle and MySQL community, be sure to also check out Sheeri Cabral’s keynote address at the 2010 MySQL User Conference for reference.


PlanetMySQL Voting: Vote UP / Vote DOWN";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}s:32:"http://purl.org/dc/elements/1.1/";a:1:{s:7:"creator";a:1:{i:0;a:5:{s:4:"data";s:15:"Ronald Bradford";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}}}i:2;a:6:{s:4:"data";s:58:" ";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";s:5:"child";a:3:{s:0:"";a:6:{s:5:"title";a:1:{i:0;a:5:{s:4:"data";s:12:"It's a girl!";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:4:"guid";a:1:{i:0;a:5:{s:4:"data";s:25:"284 at http://openlife.cc";s:7:"attribs";a:1:{s:0:"";a:1:{s:11:"isPermaLink";s:5:"false";}}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:4:"link";a:1:{i:0;a:5:{s:4:"data";s:43:"http://openlife.cc/blogs/2010/june/its-girl";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:11:"description";a:1:{i:0;a:5:{s:4:"data";s:431:"Last Saturday I became father to a baby girl - in addition to our 2½year old son. Those of you who are my former collagues from MySQL, you know about the generous Scandinavian 5 week vacations. (Which in MySQL were practiced globally.) I have decided that now is a good time for me to enjoy another Scandinavian perk: long paternity leave. I will be home with the rest of the family until approximately next February :-) read more";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:7:"pubDate";a:1:{i:0;a:5:{s:4:"data";s:31:"Tue, 08 Jun 2010 13:31:49 +0000";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:8:"category";a:4:{i:0;a:5:{s:4:"data";s:12:"MontyProgram";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:1;a:5:{s:4:"data";s:5:"MySQL";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:2;a:5:{s:4:"data";s:8:"Personal";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:3;a:5:{s:4:"data";s:7:"volcano";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}s:40:"http://purl.org/rss/1.0/modules/content/";a:1:{s:7:"encoded";a:1:{i:0;a:5:{s:4:"data";s:874:"

Last Saturday I became father to a baby girl - in addition to our 2½year old son.

Those of you who are my former collagues from MySQL, you know about the generous Scandinavian 5 week vacations. (Which in MySQL were practiced globally.) I have decided that now is a good time for me to enjoy another Scandinavian perk: long paternity leave. I will be home with the rest of the family until approximately next February :-)

read more


PlanetMySQL Voting: Vote UP / Vote DOWN";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}s:32:"http://purl.org/dc/elements/1.1/";a:1:{s:7:"creator";a:1:{i:0;a:5:{s:4:"data";s:11:"Henrik Ingo";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}}}i:3;a:6:{s:4:"data";s:63:" ";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";s:5:"child";a:3:{s:0:"";a:6:{s:5:"title";a:1:{i:0;a:5:{s:4:"data";s:42:"MariaDB 5.1.47 VMs for OpenSolaris, Ubuntu";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:4:"guid";a:1:{i:0;a:5:{s:4:"data";s:35:"http://www.bytebot.net/blog/?p=1770";s:7:"attribs";a:1:{s:0:"";a:1:{s:11:"isPermaLink";s:5:"false";}}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:4:"link";a:1:{i:0;a:5:{s:4:"data";s:100:"http://feedproxy.google.com/~r/ColinCharles/~3/ceckn8ecJHw/mariadb-5-1-47-vms-for-opensolaris-ubuntu";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:11:"description";a:1:{i:0;a:5:{s:4:"data";s:464:"Mark’s done an excellent job again, getting VM’s of MariaDB 5.1.47 (release notes, changelog) out there for download. He’s also improved the bandwidth available at the box hosting these images. Get it for Ubuntu 10.04 or OpenSolaris 0906. Related posts:Ubuntu 10.04 LTS released, MariaDB 5.1.44/5.2-BETA VM’s available VirtualBox images for MariaDB rpm -q –changelog in Debian | on IRC (or adventures in the land of #ubuntu) ";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:7:"pubDate";a:1:{i:0;a:5:{s:4:"data";s:31:"Tue, 08 Jun 2010 09:34:09 +0000";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:8:"category";a:5:{i:0;a:5:{s:4:"data";s:7:"MariaDB";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:1;a:5:{s:4:"data";s:5:"MySQL";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:2;a:5:{s:4:"data";s:11:"OpenSolaris";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:3;a:5:{s:4:"data";s:6:"Ubuntu";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:4;a:5:{s:4:"data";s:14:"Virtualization";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}s:40:"http://purl.org/rss/1.0/modules/content/";a:1:{s:7:"encoded";a:1:{i:0;a:5:{s:4:"data";s:3103:"

Mark’s done an excellent job again, getting VM’s of MariaDB 5.1.47 (release notes, changelog) out there for download. He’s also improved the bandwidth available at the box hosting these images. Get it for Ubuntu 10.04 or OpenSolaris 0906.

Related posts:

  1. Ubuntu 10.04 LTS released, MariaDB 5.1.44/5.2-BETA VM’s available
  2. VirtualBox images for MariaDB
  3. rpm -q –changelog in Debian | on IRC (or adventures in the land of #ubuntu)



PlanetMySQL Voting: Vote UP / Vote DOWN";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}s:32:"http://purl.org/dc/elements/1.1/";a:1:{s:7:"creator";a:1:{i:0;a:5:{s:4:"data";s:13:"Colin Charles";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}}}i:4;a:6:{s:4:"data";s:48:" ";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";s:5:"child";a:3:{s:0:"";a:6:{s:5:"title";a:1:{i:0;a:5:{s:4:"data";s:38:"Performance Optimization and Six Sigma";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:4:"guid";a:1:{i:0;a:5:{s:4:"data";s:43:"http://www.mysqlperformanceblog.com/?p=2958";s:7:"attribs";a:1:{s:0:"";a:1:{s:11:"isPermaLink";s:5:"false";}}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:4:"link";a:1:{i:0;a:5:{s:4:"data";s:86:"http://www.mysqlperformanceblog.com/2010/06/07/performance-optimization-and-six-sigma/";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:11:"description";a:1:{i:0;a:5:{s:4:"data";s:4014:"You might be familiar with Six Sigma business management strategy which is employed by variety of the companies in relationship to managing quality of its product. Six Sigma applies to number of defects – when you have reached six sigma quality in your production you would see 99.99966% of the products manufactured with no defects, or in other words there is less than 3 defects per million. One of principles of six sigma is what customers tend to be concerned about variance a lot more than average. For example if you produce tomato soup and the average difference from declared weight is going to be 0.1 gram or 0.5 gram, probably nobody would not notice the difference. What would worry people however is significant number of very large differences, such as half empty tomato soup can. You can see Apdex standard looking among similar lines, which is looking to classify user experiences as good, tolerable and not acceptable. So how are doing with this in MySQL (and general Web performance management) ? Not So good. In most cases organizations focus on performance rather than performance stability even though it is the later what their users really care about. Consider Adaptive Checkpoint in Innodb for example – until it got implemented we had very poor performance stability under very large number of workloads. Look at the industry standard TPC-C benchmark for example – it defines the guidance for response time as ninety five percentile response time, which is just two sigma in sigma notation. A lot of popular tools, including Sysbench and Maatkit also use 95% response time (probably inherited from these industry standards). Sometimes we use 99% response time as a guidance but this is not even quite 3 sigma which would be 99.7%. One reason we use rather weak confidence intervals in performance optimization is we need a lot of transactions to get stable measurements for stronger ones. If we’re looking at 95 percentile something like 1000 transactions will give more or less stable results in most cases. If we’re looking at 99 percentile we would need at least 10.000 and I’d look for at least 100.000 to get stable 99.9% response time. In number of my talks about performance measurements I suggested to measure 95 percentile over short period of times for example every minute, which is great for graphing and it is somewhat gives you higher confidence if you look at the whole day. I believe meeting 95 percentile response time every minute of the day is a lot harder than getting 99 percentile over 24 hours. It is worth to mention modern applications become a lot more demanding to stable performance. In the past you would have only one request to get HTML which would be responsible for most information on the page. Now with technologies like AJAX there can be number of request during page generation and interaction and each of them being slow give sluggish experience to the user. So what can we do in terms of performance management ? Start measuring much higher response time and allow for less variance. Why not to starve at least for 99.9 daily response time (one request per thousand being slow) and use 99 percentile over 5 minutes to graph response time stability. This is still very far from the confidence manufacturing uses. This is also what is probably practical for most of internet applications – A lot of applications like Gmail have 99.9 to 99.99 percent of uptime, and from the user point of view the percentile number for good response time is bound by availability figure. I also would like to see the technology vendors would focus more on the performance stability in their benchmarks. Way too often you just see the Throughput number reported which gives zero information about performance stability and so is completely irrelevant to ability of the product to provide great user experience. Entry posted by peter | 6 comments Add to: | | | | ";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:7:"pubDate";a:1:{i:0;a:5:{s:4:"data";s:31:"Mon, 07 Jun 2010 22:14:50 +0000";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:8:"category";a:2:{i:0;a:5:{s:4:"data";s:10:"benchmarks";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:1;a:5:{s:4:"data";s:5:"mysql";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}s:40:"http://purl.org/rss/1.0/modules/content/";a:1:{s:7:"encoded";a:1:{i:0;a:5:{s:4:"data";s:6486:"

You might be familiar with Six Sigma business management strategy which is employed by variety of the companies in relationship to managing quality of its product. Six Sigma applies to number of defects – when you have reached six sigma quality in your production you would see 99.99966% of the products manufactured with no defects, or in other words there is less than 3 defects per million.

One of principles of six sigma is what customers tend to be concerned about variance a lot more than average. For example if you produce tomato soup and the average difference from declared weight is going to be 0.1 gram or 0.5 gram, probably nobody would not notice the difference. What would worry people however is significant number of very large differences, such as half empty tomato soup can.

You can see Apdex standard looking among similar lines, which is looking to classify user experiences as good, tolerable and not acceptable.

So how are doing with this in MySQL (and general Web performance management) ? Not So good. In most cases organizations focus on performance rather than performance stability even though it is the later what their users really care about. Consider Adaptive Checkpoint in Innodb for example – until it got implemented we had very poor performance stability under very large number of workloads. Look at the industry standard TPC-C benchmark for example – it defines the guidance for response time as ninety five percentile response time, which is just two sigma in sigma notation. A lot of popular tools, including Sysbench and Maatkit also use 95% response time (probably inherited from these industry standards).
Sometimes we use 99% response time as a guidance but this is not even quite 3 sigma which would be 99.7%.

One reason we use rather weak confidence intervals in performance optimization is we need a lot of transactions to get stable measurements for stronger ones. If we’re looking at 95 percentile something like 1000 transactions will give more or less stable results in most cases. If we’re looking at 99 percentile we would need at least 10.000 and I’d look for at least 100.000 to get stable 99.9% response time.

In number of my talks about performance measurements I suggested to measure 95 percentile over short period of times for example every minute, which is great for graphing and it is somewhat gives you higher confidence if you look at the whole day. I believe meeting 95 percentile response time every minute of the day is a lot harder than getting 99 percentile over 24 hours.

It is worth to mention modern applications become a lot more demanding to stable performance. In the past you would have only one request to get HTML which would be responsible for most information on the page. Now with technologies like AJAX there can be number of request during page generation and interaction and each of them being slow give sluggish experience to the user.

So what can we do in terms of performance management ? Start measuring much higher response time and allow for less variance. Why not to starve at least for 99.9 daily response time (one request per thousand being slow) and use 99 percentile over 5 minutes to graph response time stability. This is still very far from the confidence manufacturing uses. This is also what is probably practical for most of internet applications – A lot of applications like Gmail have 99.9 to 99.99 percent of uptime, and from the user point of view the percentile number for good response time is bound by availability figure.

I also would like to see the technology vendors would focus more on the performance stability in their benchmarks. Way too often you just see the Throughput number reported which gives zero information about performance stability and so is completely irrelevant to ability of the product to provide great user experience.

Entry posted by peter | 6 comments

Add to: delicious | digg | reddit | netscape | Google Bookmarks


PlanetMySQL Voting: Vote UP / Vote DOWN";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}s:32:"http://purl.org/dc/elements/1.1/";a:1:{s:7:"creator";a:1:{i:0;a:5:{s:4:"data";s:22:"MySQL Performance Blog";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}}}i:5;a:6:{s:4:"data";s:58:" ";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";s:5:"child";a:3:{s:0:"";a:6:{s:5:"title";a:1:{i:0;a:5:{s:4:"data";s:15:"15 years of PHP";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:4:"guid";a:1:{i:0;a:5:{s:4:"data";s:48:"http://schlueters.de/blog/archives/136-guid.html";s:7:"attribs";a:1:{s:0:"";a:1:{s:11:"isPermaLink";s:5:"false";}}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:4:"link";a:1:{i:0;a:5:{s:4:"data";s:59:"http://schlueters.de/blog/archives/136-15-years-of-PHP.html";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:11:"description";a:1:{i:0;a:5:{s:4:"data";s:1807:"Wow, 15 years ago, on June 8th 1995, Rasmus announced PHP 1.0. Time for a personal look back: I can't remember when I first went online. Out from the local BBS systems into the wide open net. It must have been around the same time. But soon I figured out that I needed my own homepage, so I created one, using black fonts on a green (#00ff00) background. I was proud. The only issue: I had no idea how to make it accessible to others, I had the HTML file local on the PC, but well, I still was proud. I was proud since the effect of this HTML with a little bit of JavaScript was way stronger than most of my BASIC stuff I did before. A bit later, in 1998, a cousin brought a magazine about writing Perl CGI applications which caught my attention so I started learning about CGI and Perl and all the related stuff. Back then I always used my own,custom data formats for storing data. But my brother had mercy and introduced me to MySQL. I knew Paradox (and BerkeleyDB) from my Delphi programming but the powers of SQL were amazing. So I was happily building my stuff Perl+MySQL till ... yeah till the server broke down. CGI didn't work anymore for some reason. My brother, who setup our Linux box at home but didn't know much about Apache, had the pragmatic solution: PHP worked. So what did I do? - Learn PHP. And I loved it. That was in 1999. After that many tings happened. PHP evolved and became a strong player in the web market. I worked for companies like Mayflower, MySQL, Sun Microsystems and, most likely, soon Oracle. And even though I'm using way more C and C++ these days most of it is still directly related to PHP and this all due to some Greenlandic guy who published hist Personal Home Page Tools and my brother who couldn't configure CGI. 15 years and still strong. Congratulations PHP!";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:7:"pubDate";a:1:{i:0;a:5:{s:4:"data";s:31:"Mon, 07 Jun 2010 22:00:00 +0000";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:8:"category";a:4:{i:0;a:5:{s:4:"data";s:11:"anniversary";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:1;a:5:{s:4:"data";s:8:"birthday";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:2;a:5:{s:4:"data";s:6:"coding";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:3;a:5:{s:4:"data";s:3:"php";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}s:40:"http://purl.org/rss/1.0/modules/content/";a:1:{s:7:"encoded";a:1:{i:0;a:5:{s:4:"data";s:2366:"

Wow, 15 years ago, on June 8th 1995, Rasmus announced PHP 1.0. Time for a personal look back:

I can't remember when I first went online. Out from the local BBS systems into the wide open net. It must have been around the same time. But soon I figured out that I needed my own homepage, so I created one, using black fonts on a green (#00ff00) background. I was proud. The only issue: I had no idea how to make it accessible to others, I had the HTML file local on the PC, but well, I still was proud. I was proud since the effect of this HTML with a little bit of JavaScript was way stronger than most of my BASIC stuff I did before.

A bit later, in 1998, a cousin brought a magazine about writing Perl CGI applications which caught my attention so I started learning about CGI and Perl and all the related stuff. Back then I always used my own,custom data formats for storing data. But my brother had mercy and introduced me to MySQL. I knew Paradox (and BerkeleyDB) from my Delphi programming but the powers of SQL were amazing. So I was happily building my stuff Perl+MySQL till ... yeah till the server broke down. CGI didn't work anymore for some reason.

My brother, who setup our Linux box at home but didn't know much about Apache, had the pragmatic solution: PHP worked. So what did I do? - Learn PHP. And I loved it. That was in 1999. After that many tings happened. PHP evolved and became a strong player in the web market. I worked for companies like Mayflower, MySQL, Sun Microsystems and, most likely, soon Oracle. And even though I'm using way more C and C++ these days most of it is still directly related to PHP and this all due to some Greenlandic guy who published hist Personal Home Page Tools and my brother who couldn't configure CGI.

15 years and still strong. Congratulations PHP!


PlanetMySQL Voting: Vote UP / Vote DOWN";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}s:32:"http://purl.org/dc/elements/1.1/";a:1:{s:7:"creator";a:1:{i:0;a:5:{s:4:"data";s:22:"Johannes Schlüter";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}}}i:6;a:6:{s:4:"data";s:58:" ";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";s:5:"child";a:3:{s:0:"";a:6:{s:5:"title";a:1:{i:0;a:5:{s:4:"data";s:62:"Review of High Availability MySQL Cookbook by Packt Publishing";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:4:"guid";a:1:{i:0;a:5:{s:4:"data";s:59:"tag:blogger.com,1999:blog-31421954.post-5149479564300930805";s:7:"attribs";a:1:{s:0:"";a:1:{s:11:"isPermaLink";s:5:"false";}}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:4:"link";a:1:{i:0;a:5:{s:4:"data";s:75:"http://mysqldba.blogspot.com/2010/06/review-of-high-availability-mysql.html";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:11:"description";a:1:{i:0;a:5:{s:4:"data";s:1705:"A few months ago, I reviewed MySQL Admin Cookbook. Today I am reviewing High Availability MySQL Cookbook from Packt Publishing by Alex Davies. Overall, I found the book to contain some good hidden Gems.The book is a mixture of MySQL Cluster (NDB), Replication schemes, some performance tuning, some minor kernel tweaking, and some more exotic approaches to common High Availability problems. Overall, I found this book very informative and a good read.Now the specifics, the book starts out on NDB and stays focused on this fact for about 60% of the book. The next 20% is on mySQL replication then about 10% of the book is on tweaking kernel, mysql, network settings to get the most out of the system. The last 10% is a mixture of uses of exotic systems such as GFS, Conga, ISCSI and how to use these shared storage techs with mySQL.High Availability MySQL Cookbook, is a good read, and jogged my memory on NDB (since I do not use it on a day-to-day basis). If you are looking for a good reference on how to get an HA system up then this is a good book for you. If you are looking for why to use NDB over mySQL multi-master replication, this is not a book for you. The “why-to-use-this-over-that” is not the scope of the book. The drawbacks and concerns of what technology to use are not a focus of this book, and should not be. It is assumed that the reader knows which direction to go into for the most part or gives the reader enough information to set up and environment to see which way to go.In conclusion, I like this book. I think it is rather concise, and right to the point, which either gets you started into building HA systems or gives you a good reference for an existing HA environment.";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:7:"pubDate";a:1:{i:0;a:5:{s:4:"data";s:31:"Mon, 07 Jun 2010 19:07:00 +0000";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:8:"category";a:4:{i:0;a:5:{s:4:"data";s:3:"ndb";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:1;a:5:{s:4:"data";s:5:"packt";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:2;a:5:{s:4:"data";s:11:"replication";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:3;a:5:{s:4:"data";s:4:"book";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}s:40:"http://purl.org/rss/1.0/modules/content/";a:1:{s:7:"encoded";a:1:{i:0;a:5:{s:4:"data";s:2568:"
A few months ago, I reviewed MySQL Admin Cookbook. Today I am reviewing High Availability MySQL Cookbook from Packt Publishing by Alex Davies. Overall, I found the book to contain some good hidden Gems.

The book is a mixture of MySQL Cluster (NDB), Replication schemes, some performance tuning, some minor kernel tweaking, and some more exotic approaches to common High Availability problems. Overall, I found this book very informative and a good read.
Now the specifics, the book starts out on NDB and stays focused on this fact for about 60% of the book. The next 20% is on mySQL replication then about 10% of the book is on tweaking kernel, mysql, network settings to get the most out of the system. The last 10% is a mixture of uses of exotic systems such as GFS, Conga, ISCSI and how to use these shared storage techs with mySQL.

High Availability MySQL Cookbook, is a good read, and jogged my memory on NDB (since I do not use it on a day-to-day basis). If you are looking for a good reference on how to get an HA system up then this is a good book for you. If you are looking for why to use NDB over mySQL multi-master replication, this is not a book for you. The “why-to-use-this-over-that” is not the scope of the book. The drawbacks and concerns of what technology to use are not a focus of this book, and should not be. It is assumed that the reader knows which direction to go into for the most part or gives the reader enough information to set up and environment to see which way to go.

In conclusion, I like this book. I think it is rather concise, and right to the point, which either gets you started into building HA systems or gives you a good reference for an existing HA environment.

PlanetMySQL Voting: Vote UP / Vote DOWN";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}s:32:"http://purl.org/dc/elements/1.1/";a:1:{s:7:"creator";a:1:{i:0;a:5:{s:4:"data";s:17:"Dathan Pattishall";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}}}i:7;a:6:{s:4:"data";s:68:" ";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";s:5:"child";a:3:{s:0:"";a:6:{s:5:"title";a:1:{i:0;a:5:{s:4:"data";s:27:"Lock wait timeout on slaves";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:4:"guid";a:1:{i:0;a:5:{s:4:"data";s:37:"http://alexlurthu.wordpress.com/?p=69";s:7:"attribs";a:1:{s:0:"";a:1:{s:11:"isPermaLink";s:5:"false";}}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:4:"link";a:1:{i:0;a:5:{s:4:"data";s:71:"http://alexlurthu.wordpress.com/2010/06/07/lock-wait-timeout-on-slaves/";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:11:"description";a:1:{i:0;a:5:{s:4:"data";s:2616:"We had a one of our slave servers frequently stop replicating with the “Innodb Lock Wait Timeout” error. The slave IO thread would continue to fetch the binlogs while the slave SQL thread kept stopping with  the above mentioned error. The teams initial inclination was to change the innodb lock wait timeout variable from 50 secs to a higher value. It was a read-only slave. Our expectation was there would be no competing writes. Then we started listing what are the next steps possible and what could be wrong. There could be a user with “super” privilege in the system that was running updates directly on the slave A backup script that could be locking the tables out for backup Increase the “innodb lock wait timeout variable“ Enable the innodb lock monitor While we were working on the system, we noticed that there were few select queries that were running for 300+ secs of the following format. SELECT * FROM s WHERE (s.id in (select s_id from i where o_id = 79168 AND ( u_id = 79168 OR u_id = 0 )))  ORDER BY s.id DESC limit 15 We wanted to see what was innodb upto since the tables in question were innodb. The “show engine innodb status”  showed us something really interesting. —TRANSACTION 37ECF6, ACTIVE 1 sec, process no 17368, OS thread id 1263749456 fetching rows, thread declared inside InnoDB 481 mysql tables in use 2, locked 1 2242 lock struct(s), heap size 309232, 883550 row lock(s) MySQL thread id 1948263, query id 17739683 localhost root preparing SELECT * FROM s WHERE (s.id in (select s_id from i where o_id = 79168 AND ( u_id = 79168 OR u_id = 0 )))  ORDER BY s.id DESC limit 15 It was a select statement with sub query. The oddity comes from “mysql tables in use 2, locked 1“. Why does a select statement need to lock a particular table ? We replaced that particular query the with below one helped us alleviate the locking issue. SELECT * FROM  s, i WHERE s.id=i.s_id and i.o_id = 79168 and (i.u_id = 79168 OR i.u_id=0) ORDER BY s.id DESC limit 15 Innodb status output for this query : —TRANSACTION 37ECF7, ACTIVE 5 sec, process no 17368, OS thread id 1263749456 fetching rows, thread declared inside InnoDB 352 mysql tables in use 2, locked 0 MySQL thread id 1948263, query id 17739687 localhost root Sending data SELECT * FROM  s, i WHERE s.id=i.s_id and i.o_id = 79168 and (i.u_id = 79168 OR i.u_id=0) ORDER BY s.id DESC limit 15 Once the query was replaced and an index was added to o_id column, we were back to sanity. Tagged: innodb, lock wait timeout, mysql, performance, query tuning, slave ";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:7:"pubDate";a:1:{i:0;a:5:{s:4:"data";s:31:"Mon, 07 Jun 2010 13:23:08 +0000";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:8:"category";a:6:{i:0;a:5:{s:4:"data";s:5:"mysql";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:1;a:5:{s:4:"data";s:6:"innodb";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:2;a:5:{s:4:"data";s:11:"performance";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:3;a:5:{s:4:"data";s:5:"slave";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:4;a:5:{s:4:"data";s:12:"query tuning";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:5;a:5:{s:4:"data";s:17:"lock wait timeout";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}s:40:"http://purl.org/rss/1.0/modules/content/";a:1:{s:7:"encoded";a:1:{i:0;a:5:{s:4:"data";s:4647:"

We had a one of our slave servers frequently stop replicating with the “Innodb Lock Wait Timeout” error. The slave IO thread would continue to fetch the binlogs while the slave SQL thread kept stopping with  the above mentioned error. The teams initial inclination was to change the innodb lock wait timeout variable from 50 secs to a higher value. It was a read-only slave. Our expectation was there would be no competing writes. Then we started listing what are the next steps possible and what could be wrong.

  1. There could be a user with “super” privilege in the system that was running updates directly on the slave
  2. A backup script that could be locking the tables out for backup
  3. Increase the “innodb lock wait timeout variable“
  4. Enable the innodb lock monitor

While we were working on the system, we noticed that there were few select queries that were running for 300+ secs of the following format.

SELECT * FROM s WHERE (s.id in (select s_id from i where o_id = 79168 AND ( u_id = 79168 OR u_id = 0 )))  ORDER BY s.id DESC limit 15

We wanted to see what was innodb upto since the tables in question were innodb. The “show engine innodb status”  showed us something really interesting.

—TRANSACTION 37ECF6, ACTIVE 1 sec, process no 17368, OS thread id 1263749456 fetching rows, thread declared inside InnoDB 481
mysql tables in use 2, locked 1
2242 lock struct(s), heap size 309232, 883550 row lock(s)
MySQL thread id 1948263, query id 17739683 localhost root preparing
SELECT * FROM s WHERE (s.id in (select s_id from i where o_id = 79168 AND ( u_id = 79168 OR u_id = 0 )))  ORDER BY s.id DESC limit 15

It was a select statement with sub query. The oddity comes from “mysql tables in use 2, locked 1“. Why does a select statement need to lock a particular table ? We replaced that particular query the with below one helped us alleviate the locking issue.

SELECT * FROM  s, i WHERE s.id=i.s_id and i.o_id = 79168 and (i.u_id = 79168 OR i.u_id=0) ORDER BY s.id DESC limit 15

Innodb status output for this query :

—TRANSACTION 37ECF7, ACTIVE 5 sec, process no 17368, OS thread id 1263749456 fetching rows, thread declared inside InnoDB 352
mysql tables in use 2, locked 0
MySQL thread id 1948263, query id 17739687 localhost root Sending data
SELECT * FROM  s, i WHERE s.id=i.s_id and i.o_id = 79168 and (i.u_id = 79168 OR i.u_id=0) ORDER BY s.id DESC limit 15

Once the query was replaced and an index was added to o_id column, we were back to sanity.


Tagged: innodb, lock wait timeout, mysql, performance, query tuning, slave
PlanetMySQL Voting: Vote UP / Vote DOWN";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}s:32:"http://purl.org/dc/elements/1.1/";a:1:{s:7:"creator";a:1:{i:0;a:5:{s:4:"data";s:11:"Alex Lurthu";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}}}i:8;a:6:{s:4:"data";s:58:" ";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";s:5:"child";a:3:{s:0:"";a:6:{s:5:"title";a:1:{i:0;a:5:{s:4:"data";s:39:"Innodb now supports native AIO on Linux";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:4:"guid";a:1:{i:0;a:5:{s:4:"data";s:59:"tag:blogger.com,1999:blog-13606853.post-8092391481217789240";s:7:"attribs";a:1:{s:0:"";a:1:{s:11:"isPermaLink";s:5:"false";}}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:4:"link";a:1:{i:0;a:5:{s:4:"data";s:81:"http://jayant7k.blogspot.com/2010/06/innodb-now-supports-native-aio-on-linux.html";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:11:"description";a:1:{i:0;a:5:{s:4:"data";s:412:"With the exception of Windows InnoDB has used ’simulated AIO’ on all other platforms to perform certain IO operations. The IO requests that have been performed in a ’simulated AIO’ way are the write requests and the readahead requests for the datafile pages. Let us first look at what does ’simulated AIO’ mean in this context. We call it ’simulated AIO’ because it appears asynchronous from the";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:7:"pubDate";a:1:{i:0;a:5:{s:4:"data";s:31:"Mon, 07 Jun 2010 11:22:00 +0000";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:8:"category";a:4:{i:0;a:5:{s:4:"data";s:5:"mysql";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:1;a:5:{s:4:"data";s:6:"innodb";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:2;a:5:{s:4:"data";s:3:"aio";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:3;a:5:{s:4:"data";s:8:"database";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}s:40:"http://purl.org/rss/1.0/modules/content/";a:1:{s:7:"encoded";a:1:{i:0;a:5:{s:4:"data";s:625:"With the exception of Windows InnoDB has used ’simulated AIO’ on all other platforms to perform certain IO operations. The IO requests that have been performed in a ’simulated AIO’ way are the write requests and the readahead requests for the datafile pages. Let us first look at what does ’simulated AIO’ mean in this context. We call it ’simulated AIO’ because it appears asynchronous from the
PlanetMySQL Voting: Vote UP / Vote DOWN";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}s:32:"http://purl.org/dc/elements/1.1/";a:1:{s:7:"creator";a:1:{i:0;a:5:{s:4:"data";s:12:"Jayant Kumar";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}}}i:9;a:6:{s:4:"data";s:73:" ";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";s:5:"child";a:3:{s:0:"";a:6:{s:5:"title";a:1:{i:0;a:5:{s:4:"data";s:30:"change accelerator cache ratio";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:4:"guid";a:1:{i:0;a:5:{s:4:"data";s:37:"http://alexlurthu.wordpress.com/?p=64";s:7:"attribs";a:1:{s:0:"";a:1:{s:11:"isPermaLink";s:5:"false";}}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:4:"link";a:1:{i:0;a:5:{s:4:"data";s:74:"http://alexlurthu.wordpress.com/2010/06/07/change-accelerator-cache-ratio/";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:11:"description";a:1:{i:0;a:5:{s:4:"data";s:7373:"I was given the task of checking the array accelerator cache ratio and see if it was set to optimal levels. Our ideal preference was a read/write ratio of 0/100. The machine configuration is HP DL180 G5, 2 x Xeon L5420 2.50GHz, 15.7GB / 16GB 667MHz DDR2, 6 x 300GB-15K SAS.This machine was running mysql 5.1.36 using the innodb plugin. The command line utility to check the controller configuration is “hpacucli”. Navigating using hpacucli is very straight forward. “ctrl all show config detail” Will give you the entire controller configuration. => ctrl all show config detail Smart Array P400 in Slot 5 Bus Interface: PCI Slot: 5 Serial Number: P61630K9SW31NL Cache Serial Number: PA82C0J9SW02H1 RAID 6 (ADG) Status: Enabled Controller Status: OK Chassis Slot: Hardware Revision: Rev D Firmware Version: 4.12 Rebuild Priority: Medium Expand Priority: Medium Surface Scan Delay: 15 secs Post Prompt Timeout: 0 secs Cache Board Present: True Cache Status: OK Accelerator Ratio: 0% Read / 100% Write Drive Write Cache: Disabled Total Cache Size: 256 MB Battery Pack Count: 1 Battery Status: OK SATA NCQ Supported: True Array: A Interface Type: SAS Unused Space: 0 MB Status: OK Logical Drive: 1 Size: 1.4 TB Fault Tolerance: RAID 5 Heads: 255 Sectors Per Track: 32 Cylinders: 65535 Stripe Size: 64 KB Status: OK Array Accelerator: Enabled Parity Initialization Status: Initialization Completed Unique Identifier: 600508B100104B39535733314E4C0003 Disk Name: /dev/cciss/c0d0 Mount Points: / 3.9 GB, none 12.0 GB, /var 3.9 GB, /tmp 3.9 GB, /home 1.3 TB Logical Drive Label: A0432BCEP61630K9SW31NLD55F physicaldrive 1I:1:1 Port: 1I Box: 1 Bay: 1 Status: OK Drive Type: Data Drive Interface Type: SAS Size: 300 GB Rotational Speed: 15000 Firmware Revision: 0005 Serial Number: 3QP1BEP400009004UQCD Model: SEAGATE ST3300656SS PHY Count: 2 PHY Transfer Rate: 3.0GBPS, Unknown physicaldrive 1I:1:2 Port: 1I Box: 1 Bay: 2 Status: OK Drive Type: Data Drive Interface Type: SAS Size: 300 GB Rotational Speed: 15000 Firmware Revision: 0005 Serial Number: 3QP1ZZRN000090035Q2Q Model: SEAGATE ST3300656SS PHY Count: 2 PHY Transfer Rate: 3.0GBPS, Unknown physicaldrive 1I:1:3 Port: 1I Box: 1 Bay: 3 Status: OK Drive Type: Data Drive Interface Type: SAS Size: 300 GB Rotational Speed: 15000 Firmware Revision: 0005 Serial Number: 3QP20VCQ00009004XE2V Model: SEAGATE ST3300656SS PHY Count: 2 PHY Transfer Rate: 3.0GBPS, Unknown physicaldrive 1I:1:4 Port: 1I Box: 1 Bay: 4 Status: OK Drive Type: Data Drive Interface Type: SAS Size: 300 GB Rotational Speed: 15000 Firmware Revision: 0005 Serial Number: 3QP1ZZSB00009003MMWZ Model: SEAGATE ST3300656SS PHY Count: 2 PHY Transfer Rate: 3.0GBPS, Unknown physicaldrive 1I:1:5 Port: 1I Box: 1 Bay: 5 Status: OK Drive Type: Data Drive Interface Type: SAS Size: 300 GB Rotational Speed: 15000 Firmware Revision: 0005 Serial Number: 3QP1L4T000009004UQCV Model: SEAGATE ST3300656SS PHY Count: 2 PHY Transfer Rate: 3.0GBPS, Unknown physicaldrive 1I:1:6 Port: 1I Box: 1 Bay: 6 Status: OK Drive Type: Data Drive Interface Type: SAS Size: 300 GB Rotational Speed: 15000 Firmware Revision: 0005 Serial Number: 3QP196KG00009004S0ZH Model: SEAGATE ST3300656SS PHY Count: 2 PHY Transfer Rate: 3.0GBPS, Unknown In the above output our point of interest was “Accelerator Ratio: 0% Read / 100% Write“. In this case it has been set to an optimal value. In case it wasn’t set to an optimal value it can be changed using the command “ctrl slot=5 modify cacheratio=0/100“ When you are stuck for a particular command, you can just run “help <command name>” for more input. => help aa The following documentation pertains to your search: <target> modify [arrayaccelerator=enable|disable] Enables or disables the array accelerator for a given logical drive. The target can be any valid logical drive target on a controller that supports array accelerator management. <target> modify [cacheratio=#/#|?] Sets the array accelerator cache ratio for the controller. The first # is the read cache %. The second # is the write cache %. The target can be any valid controller. <target> create [type=ld] [drives=[#:]#:#,[#:]#:#,[#:]#:#-[#:]#:#],…|all|allunassigned] [raid=6|5|1+0|1|0|?] [size=#|?] [stripesize=8|16|32|64|128|256|default|?] [sectors=32|63|?] [arrayaccelerator=enable|disable|?] [drivetype=sas|satalogical|sata|saslogical|parallelscsi|?] [type=] The type parameter specifies the device type that is being created. A logical drive is the only device type supported at this time. [drives=] The drives parameter specifies the physical drives to be used for creating a logical drive on a new or existing array. If the drives specified are all unassigned drives, then a new array will be created with a new logical drive on it. If the drives specified are all assigned to an existing array, then a new logical drive will be created on that array. The symbol #:# stands for port:id or box:bay, depending on the controller. Some controllers may also support port:box:bay and use the #:#:# syntax. The all and allunassigned keywords both target all physical drives that are not currently assigned to an array. [raid=] The raid parameter sets the raid level of the logical drive. If not specified, the default raid is the highest level possible. The availability of certain raid settings depends on the number of drives designated in the “drives=” parameter. For example, RAID 1 will only be available if two drives are selected while RAID 1+0 will be shown for a selection of 4 or more drives. [size=] The size parameter specifies the size of the logical drive, the implied units are MB. If not specified, the default is the maximum possible size. [stripesize=] The stripesize parameter sets the logical drive’s stripesize. The implied units of stripe size are KB. [sectors=] The sectors parameter specifies the sectors per track of the logical drive. If not specified, the default is 32. [arrayaccelerator=] The arrayaccelerator parameter specifies the array accelerator state for the logical drive. If not specified, the default is enable. [drivetype=] The drivetype parameter specifies the drive interface type. If there are multiple drive types when selecting all physical drives, the desired drive type needs to be specified. Mixed drives are not allowed on the same array or logical drive. If all drive types in a controller are the same this parameter is not needed. The target can be a controller or an array in the system. Examples: controller slot=3 logicaldrive 2 modify arrayaccelerator=enable controller slot=1 modify cacheratio=25/75 ctrl slot=1 create type=ld drives=1:1,1:2,1:3,1:5 raid=6 ctrl slot=1 create type=ld drives=1:1-1:6,1:9,1:10-1:12 raid=6 ctrl slot=1 create type=ld drives=all drivetype=parallelscsi controller slot=5 array A create type=ld raid=5 size=1000 controller slot=1 array C create type=ld raid=1 stripesize=32 ctrl slot=1 create type=ld drives=1:1,1:2,1:3,1:5 raid=? ctrl slot=1 create type=ld drives=1:1,1:2,1:3,1:5 stripesize=? ctrl slot=1 create type=ld drives=1:1,1:2,1:3,1:5 raid=1+0 stripesize=? ctrl slot=1 create type=ld drives=1:1,1:2 raid=1 size=? ctrl slot=1 create type=ld drives=1:1,1:2,1:3,1:5 raid=1+0 sectors=? Tagged: accelerator, cache, controller, HP DL180, hpacucli, mysql ";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:7:"pubDate";a:1:{i:0;a:5:{s:4:"data";s:31:"Mon, 07 Jun 2010 10:46:47 +0000";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:8:"category";a:7:{i:0;a:5:{s:4:"data";s:8:"hardware";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:1;a:5:{s:4:"data";s:5:"mysql";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:2;a:5:{s:4:"data";s:8:"HP DL180";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:3;a:5:{s:4:"data";s:10:"controller";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:4;a:5:{s:4:"data";s:8:"hpacucli";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:5;a:5:{s:4:"data";s:11:"accelerator";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:6;a:5:{s:4:"data";s:5:"cache";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}s:40:"http://purl.org/rss/1.0/modules/content/";a:1:{s:7:"encoded";a:1:{i:0;a:5:{s:4:"data";s:10590:"

I was given the task of checking the array accelerator cache ratio and see if it was set to optimal levels. Our ideal preference was a read/write ratio of 0/100.

The machine configuration is HP DL180 G5, 2 x Xeon L5420 2.50GHz, 15.7GB / 16GB 667MHz DDR2, 6 x 300GB-15K SAS.This machine was running mysql 5.1.36 using the innodb plugin.

The command line utility to check the controller configuration is “hpacucli”. Navigating using hpacucli is very straight forward.

“ctrl all show config detail” Will give you the entire controller configuration.

=> ctrl all show config detail

Smart Array P400 in Slot 5
Bus Interface: PCI
Slot: 5
Serial Number: P61630K9SW31NL
Cache Serial Number: PA82C0J9SW02H1
RAID 6 (ADG) Status: Enabled
Controller Status: OK
Chassis Slot:
Hardware Revision: Rev D
Firmware Version: 4.12
Rebuild Priority: Medium
Expand Priority: Medium
Surface Scan Delay: 15 secs
Post Prompt Timeout: 0 secs
Cache Board Present: True
Cache Status: OK
Accelerator Ratio: 0% Read / 100% Write
Drive Write Cache: Disabled
Total Cache Size: 256 MB
Battery Pack Count: 1
Battery Status: OK
SATA NCQ Supported: True

Array: A
Interface Type: SAS
Unused Space: 0 MB
Status: OK

Logical Drive: 1
Size: 1.4 TB
Fault Tolerance: RAID 5
Heads: 255
Sectors Per Track: 32
Cylinders: 65535
Stripe Size: 64 KB
Status: OK
Array Accelerator: Enabled
Parity Initialization Status: Initialization Completed
Unique Identifier: 600508B100104B39535733314E4C0003
Disk Name: /dev/cciss/c0d0
Mount Points: / 3.9 GB, none 12.0 GB, /var 3.9 GB, /tmp 3.9 GB, /home 1.3 TB
Logical Drive Label: A0432BCEP61630K9SW31NLD55F

physicaldrive 1I:1:1
Port: 1I
Box: 1
Bay: 1
Status: OK
Drive Type: Data Drive
Interface Type: SAS
Size: 300 GB
Rotational Speed: 15000
Firmware Revision: 0005
Serial Number: 3QP1BEP400009004UQCD
Model: SEAGATE ST3300656SS
PHY Count: 2
PHY Transfer Rate: 3.0GBPS, Unknown
physicaldrive 1I:1:2
Port: 1I
Box: 1
Bay: 2
Status: OK
Drive Type: Data Drive
Interface Type: SAS
Size: 300 GB
Rotational Speed: 15000
Firmware Revision: 0005
Serial Number: 3QP1ZZRN000090035Q2Q
Model: SEAGATE ST3300656SS
PHY Count: 2
PHY Transfer Rate: 3.0GBPS, Unknown
physicaldrive 1I:1:3
Port: 1I
Box: 1
Bay: 3
Status: OK
Drive Type: Data Drive
Interface Type: SAS
Size: 300 GB
Rotational Speed: 15000
Firmware Revision: 0005
Serial Number: 3QP20VCQ00009004XE2V
Model: SEAGATE ST3300656SS
PHY Count: 2
PHY Transfer Rate: 3.0GBPS, Unknown
physicaldrive 1I:1:4
Port: 1I
Box: 1
Bay: 4
Status: OK
Drive Type: Data Drive
Interface Type: SAS
Size: 300 GB
Rotational Speed: 15000
Firmware Revision: 0005
Serial Number: 3QP1ZZSB00009003MMWZ
Model: SEAGATE ST3300656SS
PHY Count: 2
PHY Transfer Rate: 3.0GBPS, Unknown
physicaldrive 1I:1:5
Port: 1I
Box: 1
Bay: 5
Status: OK
Drive Type: Data Drive
Interface Type: SAS
Size: 300 GB
Rotational Speed: 15000
Firmware Revision: 0005
Serial Number: 3QP1L4T000009004UQCV
Model: SEAGATE ST3300656SS
PHY Count: 2
PHY Transfer Rate: 3.0GBPS, Unknown
physicaldrive 1I:1:6
Port: 1I
Box: 1
Bay: 6
Status: OK
Drive Type: Data Drive
Interface Type: SAS
Size: 300 GB
Rotational Speed: 15000
Firmware Revision: 0005
Serial Number: 3QP196KG00009004S0ZH
Model: SEAGATE ST3300656SS
PHY Count: 2
PHY Transfer Rate: 3.0GBPS, Unknown

In the above output our point of interest was “Accelerator Ratio: 0% Read / 100% Write“. In this case it has been set to an optimal value. In case it wasn’t set to an optimal value it can be changed using the command “ctrl slot=5 modify cacheratio=0/100“

When you are stuck for a particular command, you can just run “help <command name>” for more input.

=> help aa

The following documentation pertains to your search:

<target> modify [arrayaccelerator=enable|disable]
Enables or disables the array accelerator for a given logical drive. The
target can be any valid logical drive target on a controller that supports
array accelerator management.

<target> modify [cacheratio=#/#|?]
Sets the array accelerator cache ratio for the controller. The first # is
the read cache %. The second # is the write cache %. The target can be any
valid controller.

<target> create [type=ld]
[drives=[#:]#:#,[#:]#:#,[#:]#:#-[#:]#:#],…|all|allunassigned]
[raid=6|5|1+0|1|0|?]
[size=#|?]
[stripesize=8|16|32|64|128|256|default|?]
[sectors=32|63|?]
[arrayaccelerator=enable|disable|?]
[drivetype=sas|satalogical|sata|saslogical|parallelscsi|?]

[type=] The type parameter specifies the device type that is being created.
A logical drive is the only device type supported at this time.

[drives=] The drives parameter specifies the physical drives to be used for
creating a logical drive on a new or existing array. If the drives specified
are all unassigned drives, then a new array will be created with a new
logical drive on it. If the drives specified are all assigned to an existing
array, then a new logical drive will be created on that array. The symbol
#:# stands for port:id or box:bay, depending on the controller. Some
controllers may also support port:box:bay and use the #:#:# syntax. The all
and allunassigned keywords both target all physical drives that are not
currently assigned to an array.

[raid=] The raid parameter sets the raid level of the logical drive. If not
specified, the default raid is the highest level possible. The availability
of certain raid settings depends on the number of drives designated in the
“drives=” parameter. For example, RAID 1 will only be available if two
drives are selected while RAID 1+0 will be shown for a selection of 4 or
more drives.

[size=] The size parameter specifies the size of the logical drive, the
implied units are MB. If not specified, the default is the maximum possible
size.

[stripesize=] The stripesize parameter sets the logical drive’s stripesize.
The implied units of stripe size are KB.

[sectors=] The sectors parameter specifies the sectors per track of the
logical drive. If not specified, the default is 32.

[arrayaccelerator=] The arrayaccelerator parameter specifies the array
accelerator state for the logical drive. If not specified, the default is
enable.

[drivetype=] The drivetype parameter specifies the drive
interface type. If there are multiple drive types when selecting all
physical drives, the desired drive type needs to be specified. Mixed drives
are not allowed on the same array or logical drive. If all drive types in a
controller are the same this parameter is not needed. The target can be a
controller or an array in the system.

Examples:
controller slot=3 logicaldrive 2 modify arrayaccelerator=enable
controller slot=1 modify cacheratio=25/75
ctrl slot=1 create type=ld drives=1:1,1:2,1:3,1:5 raid=6
ctrl slot=1 create type=ld drives=1:1-1:6,1:9,1:10-1:12 raid=6
ctrl slot=1 create type=ld drives=all drivetype=parallelscsi
controller slot=5 array A create type=ld raid=5 size=1000
controller slot=1 array C create type=ld raid=1 stripesize=32
ctrl slot=1 create type=ld drives=1:1,1:2,1:3,1:5 raid=?
ctrl slot=1 create type=ld drives=1:1,1:2,1:3,1:5 stripesize=?
ctrl slot=1 create type=ld drives=1:1,1:2,1:3,1:5 raid=1+0 stripesize=?
ctrl slot=1 create type=ld drives=1:1,1:2 raid=1 size=?
ctrl slot=1 create type=ld drives=1:1,1:2,1:3,1:5 raid=1+0 sectors=?


Tagged: accelerator, cache, controller, HP DL180, hpacucli, mysql
PlanetMySQL Voting: Vote UP / Vote DOWN";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}s:32:"http://purl.org/dc/elements/1.1/";a:1:{s:7:"creator";a:1:{i:0;a:5:{s:4:"data";s:11:"Alex Lurthu";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}}}i:10;a:6:{s:4:"data";s:78:" ";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";s:5:"child";a:3:{s:0:"";a:6:{s:5:"title";a:1:{i:0;a:5:{s:4:"data";s:32:"InnoDB In a Complete Locked Mode";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:4:"guid";a:1:{i:0;a:5:{s:4:"data";s:64:"http://venublog.com/2010/06/07/innodb-in-a-complete-locked-mode/";s:7:"attribs";a:1:{s:0:"";a:1:{s:11:"isPermaLink";s:5:"false";}}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:4:"link";a:1:{i:0;a:5:{s:4:"data";s:64:"http://venublog.com/2010/06/07/innodb-in-a-complete-locked-mode/";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:11:"description";a:1:{i:0;a:5:{s:4:"data";s:322:"Today morning, we had a weird issue in one of the staging OLAP server (ETL); where all the InnoDB threads were locked and waiting on a signal condition for about 3-4 hours without performing any work related to InnoDB until we noticed it. Processlist indicates every thread is in ACTIVE state mode like Sending Data, [...]";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:7:"pubDate";a:1:{i:0;a:5:{s:4:"data";s:31:"Mon, 07 Jun 2010 08:42:33 +0000";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:8:"category";a:8:{i:0;a:5:{s:4:"data";s:8:"Database";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:1;a:5:{s:4:"data";s:5:"MySQL";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:2;a:5:{s:4:"data";s:12:"InnoDB hangs";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:3;a:5:{s:4:"data";s:13:"InnoDB locked";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:4;a:5:{s:4:"data";s:21:"innodb threads locked";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:5;a:5:{s:4:"data";s:26:"innodb_concurrency_tickets";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:6;a:5:{s:4:"data";s:25:"innodb_thread_concurrency";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:7;a:5:{s:4:"data";s:12:"MySQL-innodb";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}s:40:"http://purl.org/rss/1.0/modules/content/";a:1:{s:7:"encoded";a:1:{i:0;a:5:{s:4:"data";s:535:"Today morning, we had a weird issue in one of the staging OLAP server (ETL); where all the InnoDB threads were locked and waiting on a signal condition for about 3-4 hours without performing any work related to InnoDB until we noticed it. Processlist indicates every thread is in ACTIVE state mode like Sending Data, [...]
PlanetMySQL Voting: Vote UP / Vote DOWN";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}s:32:"http://purl.org/dc/elements/1.1/";a:1:{s:7:"creator";a:1:{i:0;a:5:{s:4:"data";s:13:"Venu Anuganti";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}}}i:11;a:6:{s:4:"data";s:58:" ";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";s:5:"child";a:3:{s:0:"";a:6:{s:5:"title";a:1:{i:0;a:5:{s:4:"data";s:50:"Reminder: Pacemaker/Debian webinar, today 1400 UTC";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:4:"guid";a:1:{i:0;a:5:{s:4:"data";s:34:"http://fghaas.wordpress.com/?p=530";s:7:"attribs";a:1:{s:0:"";a:1:{s:11:"isPermaLink";s:5:"false";}}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:4:"link";a:1:{i:0;a:5:{s:4:"data";s:87:"http://fghaas.wordpress.com/2010/06/07/reminder-pacemakerdebian-webinar-today-1400-utc/";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:11:"description";a:1:{i:0;a:5:{s:4:"data";s:245:"For those of you who haven’t yet registered, this is our reminder for today’s Clustering in Debian webinar at 1400 UTC. If you’re planning to run Pacemaker on the upcoming Debian squeeze release, don’t miss this! ";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:7:"pubDate";a:1:{i:0;a:5:{s:4:"data";s:31:"Mon, 07 Jun 2010 06:52:22 +0000";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:8:"category";a:4:{i:0;a:5:{s:4:"data";s:6:"Debian";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:1;a:5:{s:4:"data";s:9:"Heartbeat";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:2;a:5:{s:4:"data";s:9:"Pacemaker";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:3;a:5:{s:4:"data";s:7:"Webinar";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}s:40:"http://purl.org/rss/1.0/modules/content/";a:1:{s:7:"encoded";a:1:{i:0;a:5:{s:4:"data";s:1798:"

For those of you who haven’t yet registered, this is our reminder for today’s Clustering in Debian webinar at 1400 UTC. If you’re planning to run Pacemaker on the upcoming Debian squeeze release, don’t miss this!



PlanetMySQL Voting: Vote UP / Vote DOWN";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}s:32:"http://purl.org/dc/elements/1.1/";a:1:{s:7:"creator";a:1:{i:0;a:5:{s:4:"data";s:15:"Florian G. Haas";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}}}i:12;a:6:{s:4:"data";s:38:" ";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";s:5:"child";a:3:{s:0:"";a:5:{s:5:"title";a:1:{i:0;a:5:{s:4:"data";s:38:"1.0.6 Community Release Now Available!";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:4:"guid";a:1:{i:0;a:5:{s:4:"data";s:74:"http://infinidb.org/infinidb-blog/106-community-release-now-available.html";s:7:"attribs";a:1:{s:0:"";a:1:{s:11:"isPermaLink";s:5:"false";}}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:4:"link";a:1:{i:0;a:5:{s:4:"data";s:74:"http://infinidb.org/infinidb-blog/106-community-release-now-available.html";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:11:"description";a:1:{i:0;a:5:{s:4:"data";s:421:"We are pleased to announce the availability of the 1.0.6 release of InfiniDB Community Edition.  This is the fourth of our 1.0 monthly maintenance releases.This release includes a number of bug fixes that you can see at http://bugs.launchpad.net/infinidb.  You can download the latest InfiniDB binaries, source code, and updated documentation at: http://infinidb.org/downloads.  Read More...";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:7:"pubDate";a:1:{i:0;a:5:{s:4:"data";s:31:"Mon, 07 Jun 2010 02:15:20 +0000";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}s:40:"http://purl.org/rss/1.0/modules/content/";a:1:{s:7:"encoded";a:1:{i:0;a:5:{s:4:"data";s:661:"

We are pleased to announce the availability of the 1.0.6 release of InfiniDB Community Edition.  This is the fourth of our 1.0 monthly maintenance releases.


This release includes a number of bug fixes that you can see at http://bugs.launchpad.net/infinidb.  You can download the latest InfiniDB binaries, source code, and updated documentation at: http://infinidb.org/downloads.  Read More...
PlanetMySQL Voting: Vote UP / Vote DOWN";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}s:32:"http://purl.org/dc/elements/1.1/";a:1:{s:7:"creator";a:1:{i:0;a:5:{s:4:"data";s:31:"Calpont & InfiniDB Project Team";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}}}i:13;a:6:{s:4:"data";s:38:" ";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";s:5:"child";a:3:{s:0:"";a:5:{s:5:"title";a:1:{i:0;a:5:{s:4:"data";s:33:"1.1.3 Beta Release Now Available!";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:4:"guid";a:1:{i:0;a:5:{s:4:"data";s:69:"http://infinidb.org/infinidb-blog/113-beta-release-now-available.html";s:7:"attribs";a:1:{s:0:"";a:1:{s:11:"isPermaLink";s:5:"false";}}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:4:"link";a:1:{i:0;a:5:{s:4:"data";s:69:"http://infinidb.org/infinidb-blog/113-beta-release-now-available.html";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:11:"description";a:1:{i:0;a:5:{s:4:"data";s:497:"We are pleased to announce the availability of the 1.1.3 Beta release of InfiniDB Community Edition.  This is our initial beta release for 1.1 and it is not recommended for production work.This release delivers the final content for our 1.1 release, thus we have moved from an Alpha designation to a Beta designation.  We will be putting out weekly releases in June containing bug fixes and performance tweaks as we march towards our 1.1 Final release.&nbRead More...";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:7:"pubDate";a:1:{i:0;a:5:{s:4:"data";s:31:"Mon, 07 Jun 2010 01:44:21 +0000";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}s:40:"http://purl.org/rss/1.0/modules/content/";a:1:{s:7:"encoded";a:1:{i:0;a:5:{s:4:"data";s:725:"

We are pleased to announce the availability of the 1.1.3 Beta release of InfiniDB Community Edition.  This is our initial beta release for 1.1 and it is not recommended for production work.

This release delivers the final content for our 1.1 release, thus we have moved from an Alpha designation to a Beta designation.  We will be putting out weekly releases in June containing bug fixes and performance tweaks as we march towards our 1.1 Final release.&nbRead More...
PlanetMySQL Voting: Vote UP / Vote DOWN";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}s:32:"http://purl.org/dc/elements/1.1/";a:1:{s:7:"creator";a:1:{i:0;a:5:{s:4:"data";s:31:"Calpont & InfiniDB Project Team";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}}}i:14;a:6:{s:4:"data";s:38:" ";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";s:5:"child";a:3:{s:0:"";a:5:{s:5:"title";a:1:{i:0;a:5:{s:4:"data";s:27:"Oracle's MySQL Report Card,";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:4:"guid";a:1:{i:0;a:5:{s:4:"data";s:58:"tag:blogger.com,1999:blog-27751205.post-913921763537755316";s:7:"attribs";a:1:{s:0:"";a:1:{s:11:"isPermaLink";s:5:"false";}}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:4:"link";a:1:{i:0;a:5:{s:4:"data";s:66:"http://antbits.blogspot.com/2010/06/oracles-mysql-report-card.html";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:11:"description";a:1:{i:0;a:5:{s:4:"data";s:396:"Yeah, I know it is early days right now but I do think that Oracle's acquisition of Sun Microsystems has not harmed MySQL.Oracle has not done anything to get in the way of MySQL and as far as I can tell, it's basically business as normal: People are still using MySQL.Admittedly, I have been busy at my job at Blizzard Entertainment but I really haven't seen anything which alarms me.Besides, the";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:7:"pubDate";a:1:{i:0;a:5:{s:4:"data";s:31:"Sun, 06 Jun 2010 23:55:03 +0000";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}s:40:"http://purl.org/rss/1.0/modules/content/";a:1:{s:7:"encoded";a:1:{i:0;a:5:{s:4:"data";s:609:"Yeah, I know it is early days right now but I do think that Oracle's acquisition of Sun Microsystems has not harmed MySQL.Oracle has not done anything to get in the way of MySQL and as far as I can tell, it's basically business as normal: People are still using MySQL.Admittedly, I have been busy at my job at Blizzard Entertainment but I really haven't seen anything which alarms me.Besides, the
PlanetMySQL Voting: Vote UP / Vote DOWN";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}s:32:"http://purl.org/dc/elements/1.1/";a:1:{s:7:"creator";a:1:{i:0;a:5:{s:4:"data";s:15:"Antony T Curtis";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}}}i:15;a:6:{s:4:"data";s:63:" ";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";s:5:"child";a:3:{s:0:"";a:6:{s:5:"title";a:1:{i:0;a:5:{s:4:"data";s:43:"Performance gain of MySQL 5.1 InnoDB plugin";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:4:"guid";a:1:{i:0;a:5:{s:4:"data";s:59:"tag:blogger.com,1999:blog-16959946.post-2434285055672137838";s:7:"attribs";a:1:{s:0:"";a:1:{s:11:"isPermaLink";s:5:"false";}}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:4:"link";a:1:{i:0;a:5:{s:4:"data";s:80:"http://datacharmer.blogspot.com/2010/06/performance-gain-of-mysql-51-innodb.html";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:11:"description";a:1:{i:0;a:5:{s:4:"data";s:2218:"You know already that InnoDB in MySQL 5.5 has great improvements in performance and scalability. You will have to wait a few months for that, though, because MySQL 5.5 is not GA yet. But if you need some extra performance in MySQL 5.1, you may want to use the Innodb Plugin instead of the built-in one. As of version 5.1.47, the Innodb plugin is of GA quality, and it comes with a good out-of-the-box improvement compared to the built-in engine.To test my assumptions, I used one of my test Linux servers to perform a sysbench on 5.0.91, 5.1.47 built-in and plugin, and 5.5.4. The MySQL servers were all configured with innodb_buffer_pool_size=5GMySQL 4.1.47 was tested both as out-of-the-box, and with the plugin enabled.ignore_builtin_innodb# note: the following statements must go all in one lineplugin-load=innodb=ha_innodb_plugin.so;innodb_trx=ha_innodb_plugin.so;innodb_locks=ha_innodb_plugin.so;innodb_lock_waits=ha_innodb_plugin.so;innodb_cmp=ha_innodb_plugin.so;innodb_cmp_reset=ha_innodb_plugin.so;innodb_cmpmem=ha_innodb_plugin.so;innodb_cmpmem_reset=ha_innodb_plugin.sodefault-storage-engine=InnoDBinnodb_file_per_table=1innodb_file_format=barracudainnodb_strict_mode=1The test was the same for all the servers. A simple sysbench both read-only and read/write on a 1M records table.sysbench \ --test=oltp \ --oltp-table-size=1000000 \ --mysql-db=test \ --mysql-user=$USER \ --mysql-password=$PASSWD \ --mysql-host=$HOST \ --mysql-port=$PORT \ --max-time=60 \ --oltp-read-only=$ON_OFF \ --max-requests=0 \ --num-threads=8 runWhat came out is that, by using the innodb plugin instead of the built-in engine, you get roughly 15% more in read-only, and close to 8% in read/write.Note that 5.5. enhancements are more impressive in scalability tests with more than 8 cores. In this server, I have just tested a simple scenario.I did some more testing using "ROW_FORMAT=COMPRESSED KEY_BLOCK_SIZE=X" in the InnoDB table, where X changed from 4 to 16. But sysbench didn't seem to play well with compression. For low values of KEY_BLOCK_SIZE, you actually get a much worse result than the built-in engine. I have yet to figure out how I would use this compressed InnoDB in practice.";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:7:"pubDate";a:1:{i:0;a:5:{s:4:"data";s:31:"Sun, 06 Jun 2010 19:30:00 +0000";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:8:"category";a:5:{i:0;a:5:{s:4:"data";s:11:"performance";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:1;a:5:{s:4:"data";s:5:"mysql";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:2;a:5:{s:4:"data";s:6:"plugin";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:3;a:5:{s:4:"data";s:6:"innodb";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:4;a:5:{s:4:"data";s:7:"testing";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}s:40:"http://purl.org/rss/1.0/modules/content/";a:1:{s:7:"encoded";a:1:{i:0;a:5:{s:4:"data";s:3588:"
plugin performanceYou know already that InnoDB in MySQL 5.5 has great improvements in performance and scalability. You will have to wait a few months for that, though, because MySQL 5.5 is not GA yet.
But if you need some extra performance in MySQL 5.1, you may want to use the Innodb Plugin instead of the built-in one. As of version 5.1.47, the Innodb plugin is of GA quality, and it comes with a good out-of-the-box improvement compared to the built-in engine.

To test my assumptions, I used one of my test Linux servers to perform a sysbench on 5.0.91, 5.1.47 built-in and plugin, and 5.5.4. The MySQL servers were all configured with 

innodb_buffer_pool_size=5G

MySQL 4.1.47 was tested both as out-of-the-box, and with the plugin enabled.

ignore_builtin_innodb
# note: the following statements must go all in one line
plugin-load=innodb=ha_innodb_plugin.so;innodb_trx=ha_innodb_plugin.so;innodb_locks=ha_innodb_plugin.so;innodb_lock_waits=ha_innodb_plugin.so;innodb_cmp=ha_innodb_plugin.so;innodb_cmp_reset=ha_innodb_plugin.so;innodb_cmpmem=ha_innodb_plugin.so;innodb_cmpmem_reset=ha_innodb_plugin.so

default-storage-engine=InnoDBinnodb_file_per_table=1
innodb_file_format=barracudainnodb_strict_mode=1

The test was the same for all the servers. A simple sysbench both read-only and read/write on a 1M records table.

sysbench \
    --test=oltp \
    --oltp-table-size=1000000 \
    --mysql-db=test \
    --mysql-user=$USER \
    --mysql-password=$PASSWD \
    --mysql-host=$HOST \
    --mysql-port=$PORT \
    --max-time=60 \
    --oltp-read-only=$ON_OFF \
    --max-requests=0 \
    --num-threads=8 run

What came out is that, by using the innodb plugin instead of the built-in engine, you get roughly 15% more in read-only, and close to 8% in read/write.


Note that 5.5. enhancements are more impressive in scalability tests with more than 8 cores. In this server, I have just tested a simple scenario.

I did some more testing using "ROW_FORMAT=COMPRESSED KEY_BLOCK_SIZE=X" in the InnoDB table, where X changed from 4 to 16. But sysbench didn't seem to play well with compression. For low values of KEY_BLOCK_SIZE, you actually get a much worse result than the built-in engine. I have yet to figure out how I would use this compressed InnoDB in practice.


PlanetMySQL Voting: Vote UP / Vote DOWN";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}s:32:"http://purl.org/dc/elements/1.1/";a:1:{s:7:"creator";a:1:{i:0;a:5:{s:4:"data";s:14:"Giuseppe Maxia";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}}}i:16;a:6:{s:4:"data";s:53:" ";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";s:5:"child";a:3:{s:0:"";a:6:{s:5:"title";a:1:{i:0;a:5:{s:4:"data";s:43:"Why is MySQL documentation search so wrong?";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:4:"guid";a:1:{i:0;a:5:{s:4:"data";s:38:"http://ronaldbradford.com/blog/?p=2864";s:7:"attribs";a:1:{s:0:"";a:1:{s:11:"isPermaLink";s:5:"false";}}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:4:"link";a:1:{i:0;a:5:{s:4:"data";s:85:"http://ronaldbradford.com/blog/why-is-mysql-documentation-search-so-wrong-2010-06-06/";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:11:"description";a:1:{i:0;a:5:{s:4:"data";s:407:"I just don’t get this I don not know what the technology is behind the search box at MySQL Documentation but it annoys me when I want to see the syntax of a command and Search can’t find the page, when a dedicated page exists and I’m using the exact syntax of both the command the title of the page. In this example the search was for SHOW WARNINGS. Google find the page as first link! ";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:7:"pubDate";a:1:{i:0;a:5:{s:4:"data";s:31:"Sun, 06 Jun 2010 18:21:13 +0000";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:8:"category";a:3:{i:0;a:5:{s:4:"data";s:9:"Databases";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:1;a:5:{s:4:"data";s:5:"MySQL";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:2;a:5:{s:4:"data";s:12:"Professional";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}s:40:"http://purl.org/rss/1.0/modules/content/";a:1:{s:7:"encoded";a:1:{i:0;a:5:{s:4:"data";s:956:"

I just don’t get this I don not know what the technology is behind the search box at MySQL Documentation but it annoys me when I want to see the syntax of a command and Search can’t find the page, when a dedicated page exists and I’m using the exact syntax of both the command the title of the page. In this example the search was for SHOW WARNINGS.

Google find the page as first link!


PlanetMySQL Voting: Vote UP / Vote DOWN";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}s:32:"http://purl.org/dc/elements/1.1/";a:1:{s:7:"creator";a:1:{i:0;a:5:{s:4:"data";s:15:"Ronald Bradford";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}}}i:17;a:6:{s:4:"data";s:53:" ";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";s:5:"child";a:2:{s:0:"";a:6:{s:5:"title";a:1:{i:0;a:5:{s:4:"data";s:69:"Installing Apache2 With PHP5 And MySQL Support On Ubuntu 10.04 (LAMP)";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:4:"guid";a:1:{i:0;a:5:{s:4:"data";s:93:"http://www.howtoforge.com/installing-apache2-with-php5-and-mysql-support-on-ubuntu-10.04-lamp";s:7:"attribs";a:1:{s:0:"";a:1:{s:11:"isPermaLink";s:5:"false";}}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:4:"link";a:1:{i:0;a:5:{s:4:"data";s:93:"http://www.howtoforge.com/installing-apache2-with-php5-and-mysql-support-on-ubuntu-10.04-lamp";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:11:"description";a:1:{i:0;a:5:{s:4:"data";s:257:" Installing Apache2 With PHP5 And MySQL Support On Ubuntu 10.04 (LAMP) LAMP is short for Linux, Apache, MySQL, PHP. This tutorial shows how you can install an Apache2 webserver on an Ubuntu 10.04 server with PHP5 support (mod_php) and MySQL support.";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:7:"pubDate";a:1:{i:0;a:5:{s:4:"data";s:31:"Sun, 06 Jun 2010 16:50:39 +0000";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:8:"category";a:4:{i:0;a:5:{s:4:"data";s:6:"Ubuntu";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:1;a:5:{s:4:"data";s:6:"Apache";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:2;a:5:{s:4:"data";s:5:"MySQL";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:3;a:5:{s:4:"data";s:3:"PHP";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}s:40:"http://purl.org/rss/1.0/modules/content/";a:1:{s:7:"encoded";a:1:{i:0;a:5:{s:4:"data";s:755:"

Installing Apache2 With PHP5 And MySQL Support On Ubuntu 10.04 (LAMP)

LAMP is short for Linux, Apache, MySQL, PHP. This tutorial shows how you can install an Apache2 webserver on an Ubuntu 10.04 server with PHP5 support (mod_php) and MySQL support.


PlanetMySQL Voting: Vote UP / Vote DOWN";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}}}i:18;a:6:{s:4:"data";s:83:" ";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";s:5:"child";a:3:{s:0:"";a:6:{s:5:"title";a:1:{i:0;a:5:{s:4:"data";s:35:"I’ll have a MySQL shot to go!";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:4:"guid";a:1:{i:0;a:5:{s:4:"data";s:38:"http://ronaldbradford.com/blog/?p=2857";s:7:"attribs";a:1:{s:0:"";a:1:{s:11:"isPermaLink";s:5:"false";}}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:4:"link";a:1:{i:0;a:5:{s:4:"data";s:70:"http://ronaldbradford.com/blog/ill-have-a-mysql-shot-to-go-2010-06-06/";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:11:"description";a:1:{i:0;a:5:{s:4:"data";s:339:"Wednesday night of the MySQL track of ODTUG Kaleidoscope will include an evening with Last Comic Standing comedian, John Heffron. It should be great way to unwind after day 3 of the conference. Black vodka anybody. Check out the MySQL Schedule for more information of presentations for the 4 days. More details is also available here.";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:7:"pubDate";a:1:{i:0;a:5:{s:4:"data";s:31:"Sun, 06 Jun 2010 15:36:45 +0000";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:8:"category";a:9:{i:0;a:5:{s:4:"data";s:9:"Databases";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:1;a:5:{s:4:"data";s:17:"Kaleidoscope 2010";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:2;a:5:{s:4:"data";s:5:"MySQL";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:3;a:5:{s:4:"data";s:6:"Oracle";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:4;a:5:{s:4:"data";s:24:"Oracle/MySQL Conferences";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:5;a:5:{s:4:"data";s:12:"Professional";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:6;a:5:{s:4:"data";s:12:"john heffron";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:7;a:5:{s:4:"data";s:12:"Kaleidoscope";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:8;a:5:{s:4:"data";s:5:"odtug";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}s:40:"http://purl.org/rss/1.0/modules/content/";a:1:{s:7:"encoded";a:1:{i:0;a:5:{s:4:"data";s:890:"

Wednesday night of the MySQL track of ODTUG Kaleidoscope will include an evening with Last Comic Standing comedian, John Heffron. It should be great way to unwind after day 3 of the conference. Black vodka anybody.

Check out the MySQL Schedule for more information of presentations for the 4 days. More details is also available here.


PlanetMySQL Voting: Vote UP / Vote DOWN";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}s:32:"http://purl.org/dc/elements/1.1/";a:1:{s:7:"creator";a:1:{i:0;a:5:{s:4:"data";s:15:"Ronald Bradford";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}}}i:19;a:6:{s:4:"data";s:38:" ";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";s:5:"child";a:3:{s:0:"";a:5:{s:5:"title";a:1:{i:0;a:5:{s:4:"data";s:43:"Flexviews and FlexCDC 1.5.3beta is released";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:4:"guid";a:1:{i:0;a:5:{s:4:"data";s:43:"http://swanhart.livejournal.com/133312.html";s:7:"attribs";a:1:{s:0:"";a:1:{s:11:"isPermaLink";s:5:"false";}}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:4:"link";a:1:{i:0;a:5:{s:4:"data";s:43:"http://swanhart.livejournal.com/133312.html";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:11:"description";a:1:{i:0;a:5:{s:4:"data";s:1762:"Improvements for Flexviews, materialized view for MySQL 5.1+There is up-to-date documentation available as manual.html, a robodoc generated version of the manual for the SQL_API interface. The manual is generated from RoboDOC comments embedded in the source code. This makes it easy to maintain the documentation if the interface changes.Flexviews has a new easier to read, and hopefully easier to understand website. You can find a copy of the manual on the web site as well.There is a SQL converter on the website than will automatically generate Flexviews SQL_API calls from a SELECT statement. The next step will be to turn this into a MySQL proxy script that can handle 'CREATE ALGORITHM=INCREMENTAL MATERIALIZED VIEW ...'Support for the FlexCDC auto_changelog mode, which automatically logs changes of all tables. This is great for small schemas.Improvements for FlexCDCThe main wrapper scripts around flexcdc.php accidently included 'consumer.php' instead of 'flexcdc.php', which I had renamed shortly before the prior release. This problem has been resolved.I added an ini parameter to control the issuing of warnings by the scriptI added info to the README about when to not use auto_changelogflexcdc.php now looks for the ini file in the same directory in which the script resides, instead of the working directorythe next release will have a command line option to specify the ini location. this will make it easy to run more than one consumer.I am going to work on a HOWTO video which shows how to install and use Flexviews. I really hope that the automatic conversion of SQL by the conversion tool helps people understand the API. For those who still don't want to use it, well I guess you will have to wait for the MySQL proxy integration.";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:7:"pubDate";a:1:{i:0;a:5:{s:4:"data";s:31:"Sun, 06 Jun 2010 01:19:54 +0000";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}s:40:"http://purl.org/rss/1.0/modules/content/";a:1:{s:7:"encoded";a:1:{i:0;a:5:{s:4:"data";s:2373:"

Improvements for Flexviews, materialized view for MySQL 5.1+



  1. There is up-to-date documentation available as manual.html, a robodoc generated version of the manual for the SQL_API interface. The manual is generated from RoboDOC comments embedded in the source code. This makes it easy to maintain the documentation if the interface changes.
  2. Flexviews has a new easier to read, and hopefully easier to understand website. You can find a copy of the manual on the web site as well.
  3. There is a SQL converter on the website than will automatically generate Flexviews SQL_API calls from a SELECT statement. The next step will be to turn this into a MySQL proxy script that can handle 'CREATE ALGORITHM=INCREMENTAL MATERIALIZED VIEW ...'
  4. Support for the FlexCDC auto_changelog mode, which automatically logs changes of all tables. This is great for small schemas.


Improvements for FlexCDC



  1. The main wrapper scripts around flexcdc.php accidently included 'consumer.php' instead of 'flexcdc.php', which I had renamed shortly before the prior release. This problem has been resolved.
  2. I added an ini parameter to control the issuing of warnings by the script
  3. I added info to the README about when to not use auto_changelog
  4. flexcdc.php now looks for the ini file in the same directory in which the script resides, instead of the working directory
  5. the next release will have a command line option to specify the ini location. this will make it easy to run more than one consumer.


I am going to work on a HOWTO video which shows how to install and use Flexviews. I really hope that the automatic conversion of SQL by the conversion tool helps people understand the API. For those who still don't want to use it, well I guess you will have to wait for the MySQL proxy integration.
PlanetMySQL Voting: Vote UP / Vote DOWN";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}s:32:"http://purl.org/dc/elements/1.1/";a:1:{s:7:"creator";a:1:{i:0;a:5:{s:4:"data";s:15:"Justin Swanhart";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}}}i:20;a:6:{s:4:"data";s:68:" ";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";s:5:"child";a:3:{s:0:"";a:6:{s:5:"title";a:1:{i:0;a:5:{s:4:"data";s:49:"Software patents are a bad legacy to leave behind";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:4:"guid";a:1:{i:0;a:5:{s:4:"data";s:25:"283 at http://openlife.cc";s:7:"attribs";a:1:{s:0:"";a:1:{s:11:"isPermaLink";s:5:"false";}}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:4:"link";a:1:{i:0;a:5:{s:4:"data";s:79:"http://openlife.cc/blogs/2010/june/software-patents-are-bad-legacy-leave-behind";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:11:"description";a:1:{i:0;a:5:{s:4:"data";s:394:"Glyn Moody has an interesting piece on Why Patents are Like Black Holes where he looks at the situation when a large patent holder goes bankrupt - or is about to. His point is that even if a company otherwise can go out of business cleanly, the patents often remain as a piece of "IPR" that can come back and haunt us like a zombie. Also Matt Asay recently weighed in on the subject: read more";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:7:"pubDate";a:1:{i:0;a:5:{s:4:"data";s:31:"Sat, 05 Jun 2010 07:44:32 +0000";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:8:"category";a:6:{i:0;a:5:{s:4:"data";s:6:"Ethics";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:1;a:5:{s:4:"data";s:5:"MySQL";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:2;a:5:{s:4:"data";s:6:"Oracle";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:3;a:5:{s:4:"data";s:7:"Patents";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:4;a:5:{s:4:"data";s:8:"Politics";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:5;a:5:{s:4:"data";s:3:"Sun";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}s:40:"http://purl.org/rss/1.0/modules/content/";a:1:{s:7:"encoded";a:1:{i:0;a:5:{s:4:"data";s:922:"

Glyn Moody has an interesting piece on Why Patents are Like Black Holes where he looks at the situation when a large patent holder goes bankrupt - or is about to. His point is that even if a company otherwise can go out of business cleanly, the patents often remain as a piece of "IPR" that can come back and haunt us like a zombie.

Also Matt Asay recently weighed in on the subject:

read more


PlanetMySQL Voting: Vote UP / Vote DOWN";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}s:32:"http://purl.org/dc/elements/1.1/";a:1:{s:7:"creator";a:1:{i:0;a:5:{s:4:"data";s:11:"Henrik Ingo";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}}}i:21;a:6:{s:4:"data";s:48:" ";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";s:5:"child";a:3:{s:0:"";a:6:{s:5:"title";a:1:{i:0;a:5:{s:4:"data";s:33:"i have other affairs to attend to";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:4:"guid";a:1:{i:0;a:5:{s:4:"data";s:37:"tag:trainedmonkey.com,2010-06-04:3199";s:7:"attribs";a:1:{s:0:"";a:1:{s:11:"isPermaLink";s:5:"false";}}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:4:"link";a:1:{i:0;a:5:{s:4:"data";s:67:"http://trainedmonkey.com/2010/6/4/i_have_other_affairs_to_attend_to";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:11:"description";a:1:{i:0;a:5:{s:4:"data";s:130:"my last day with oracle (formerly sun, formerly mysql), today, came a little over eight years after my first.on to the next thing.";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:7:"pubDate";a:1:{i:0;a:5:{s:4:"data";s:31:"Sat, 05 Jun 2010 01:53:25 +0000";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:8:"category";a:2:{i:0;a:5:{s:4:"data";s:5:"mysql";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:1;a:5:{s:4:"data";s:8:"personal";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}s:40:"http://purl.org/rss/1.0/modules/content/";a:1:{s:7:"encoded";a:1:{i:0;a:5:{s:4:"data";s:355:"my last day with oracle (formerly sun, formerly mysql), today, came a little over eight years after my first.

on to the next thing.
PlanetMySQL Voting: Vote UP / Vote DOWN";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}s:32:"http://purl.org/dc/elements/1.1/";a:1:{s:7:"creator";a:1:{i:0;a:5:{s:4:"data";s:12:"Jim Winstead";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}}}i:22;a:6:{s:4:"data";s:78:" ";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";s:5:"child";a:3:{s:0:"";a:6:{s:5:"title";a:1:{i:0;a:5:{s:4:"data";s:52:"Log Buffer #191, A Carnival of the Vanities for DBAs";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:4:"guid";a:1:{i:0;a:5:{s:4:"data";s:36:"http://www.pythian.com/news/?p=12977";s:7:"attribs";a:1:{s:0:"";a:1:{s:11:"isPermaLink";s:5:"false";}}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:4:"link";a:1:{i:0;a:5:{s:4:"data";s:85:"http://www.pythian.com/news/12977/log-buffer-191-a-carnival-of-the-vanities-for-dbas/";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:11:"description";a:1:{i:0;a:5:{s:4:"data";s:2007:"Welcome to Log Buffer, the weekly roundup of database blogs. Kicking off this week in Log Buffer #191 are posts from Alisher Yuldashev: Randolf Geist blogs on an Advanced Oracle Troubleshooting Session – PGA/UGA memory fragmentation for when a batch process takes significantly longer than expected. James Morle talks about an example of a misleading average in Log File Sync and AWR – Not Good Bedfellows. And a few faves from Bradd Piontek: Marco Gralike, on Blog.Gralike.Com, revisits Enabling and Disabling Database Options, a small item that is easily overlooked. Marco also notes a cool tool: VirtualBox Appliance which makes a great start-up test environment. Word of caution however, it’s for testing purposes only. On Askdba.org, Amit advises on downloading Oracle software directly to server in a post based on Pythian’s downloading from OTN directly to your database server. Watch for future posts from Brad on how he does it via Firefox, and edelivery.oracle.com. Alex Gorbachev is spreading the word about The Ultimate SQL Tune-off with Jonathan Lewis and Kyle Hailey, two of his most respected Oracle performance experts, believing the session should be interesting to all DBAs, not just Oracle. Robert Catteral continues to recap session highlights from the International DB2 Users Group Conference last month in Nuggets from DB2 by the Bay, Part 3, following Parts 1 & 2. Chen Shapira contributed Cloning Oracle Home from RAC to Stand-Alone. On In Recovery, Paul Randal wrote the whitepaper Proven SQL Server Architectures for High Availability and Disaster Recovery he wrote for the Spring SQL Server release has been published. Moving to MySQL world, Vadim Tkachenko continues storage benchmarking of MySQL FlashCache (very much like Oracle FlashCache but for MySQL InnoDB engine). This time he is using FusionIO cards for FlashCache. And, to round things off, Ronald Bradford writes about When SET GLOBAL affects SESSION scope. Have a great weekend everyone.";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:7:"pubDate";a:1:{i:0;a:5:{s:4:"data";s:31:"Fri, 04 Jun 2010 20:54:55 +0000";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:8:"category";a:8:{i:0;a:5:{s:4:"data";s:10:"Log Buffer";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:1;a:5:{s:4:"data";s:5:"MySQL";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:2;a:5:{s:4:"data";s:5:"NoSQL";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:3;a:5:{s:4:"data";s:6:"Oracle";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:4;a:5:{s:4:"data";s:10:"PostgreSQL";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:5;a:5:{s:4:"data";s:10:"SQL Server";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:6;a:5:{s:4:"data";s:14:"Technical Blog";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:7;a:5:{s:4:"data";s:4:"blog";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}s:40:"http://purl.org/rss/1.0/modules/content/";a:1:{s:7:"encoded";a:1:{i:0;a:5:{s:4:"data";s:4972:"

Welcome to Log Buffer, the weekly roundup of database blogs.

Kicking off this week in Log Buffer #191 are posts from Alisher Yuldashev:

Randolf Geist blogs on an Advanced Oracle Troubleshooting Session – PGA/UGA memory fragmentation for when a batch process takes significantly longer than expected.

James Morle talks about an example of a misleading average in Log File Sync and AWR – Not Good Bedfellows.

And a few faves from Bradd Piontek:

Marco Gralike, on Blog.Gralike.Com, revisits Enabling and Disabling Database Options, a small item that is easily overlooked. Marco also notes a cool tool: VirtualBox Appliance which makes a great start-up test environment. Word of caution however, it’s for testing purposes only.

On Askdba.org, Amit advises on downloading Oracle software directly to server in a post based on Pythian’s downloading from OTN directly to your database server. Watch for future posts from Brad on how he does it via Firefox, and edelivery.oracle.com.

Alex Gorbachev is spreading the word about The Ultimate SQL Tune-off with Jonathan Lewis and Kyle Hailey, two of his most respected Oracle performance experts, believing the session should be interesting to all DBAs, not just Oracle.

Robert Catteral continues to recap session highlights from the International DB2 Users Group Conference last month in Nuggets from DB2 by the Bay, Part 3, following Parts 1 & 2.

Chen Shapira contributed Cloning Oracle Home from RAC to Stand-Alone.

On In Recovery, Paul Randal wrote the whitepaper Proven SQL Server Architectures for High Availability and Disaster Recovery he wrote for the Spring SQL Server release has been published.

Moving to MySQL world, Vadim Tkachenko continues storage benchmarking of MySQL FlashCache (very much like Oracle FlashCache but for MySQL InnoDB engine). This time he is using FusionIO cards for FlashCache.

And, to round things off, Ronald Bradford writes about When SET GLOBAL affects SESSION scope.

Have a great weekend everyone.


PlanetMySQL Voting: Vote UP / Vote DOWN";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}s:32:"http://purl.org/dc/elements/1.1/";a:1:{s:7:"creator";a:1:{i:0;a:5:{s:4:"data";s:17:"The Pythian Group";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}}}i:23;a:6:{s:4:"data";s:53:" ";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";s:5:"child";a:3:{s:0:"";a:6:{s:5:"title";a:1:{i:0;a:5:{s:4:"data";s:54:"What's New in the MySQL Enterprise Spring 2010 Release";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:4:"guid";a:1:{i:0;a:5:{s:4:"data";s:69:"tag:blogger.com,1999:blog-8877901999053801110.post-904615822765297343";s:7:"attribs";a:1:{s:0:"";a:1:{s:11:"isPermaLink";s:5:"false";}}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:4:"link";a:1:{i:0;a:5:{s:4:"data";s:113:"http://feedproxy.google.com/~r/ItsJustAboutCommunication/~3/CudIeD0sST0/whats-new-in-mysql-enterprise-spring.html";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:11:"description";a:1:{i:0;a:5:{s:4:"data";s:1225:"The MySQL Enterprise Spring 2010 Release includes key improvements designed to help you proactively optimize MySQL performance and identify potential issues before they can become costly outages or slowdowns. In this webinar we will explore the new features of the MySQL Enterprise Monitor 2.2 that are designed to save DBAs and Developers time and effort in keeping MySQL systems running at the highest levels of security, performance and availability. Join us and learn about: How the MySQL Connectors can now be used to collect, aggregate and monitor MySQL query performance data How to quickly identify which queries are performing full table scans How to isolate your worst performing queries based on total execution counts, time, data returned, date/time, query content, query type, and more How to identify and drill down into line-level source code for specific query executions New LDAP authentication model New Read Only User New Auto-Closing Events New MySQL Support Diagnostic Reports New Custom Rule/Graph Import and Export options New Faster Purging of Repository Data New Installation Options And more! Here are the links for the various localized webinars:English, Italian, German, French Luca ";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:7:"pubDate";a:1:{i:0;a:5:{s:4:"data";s:31:"Fri, 04 Jun 2010 13:49:00 +0000";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:8:"category";a:3:{i:0;a:5:{s:4:"data";s:5:"mysql";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:1;a:5:{s:4:"data";s:10:"enterprise";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:2;a:5:{s:4:"data";s:8:"webinars";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}s:40:"http://purl.org/rss/1.0/modules/content/";a:1:{s:7:"encoded";a:1:{i:0;a:5:{s:4:"data";s:4604:"The MySQL Enterprise Spring 2010 Release includes key improvements designed to help you proactively optimize MySQL performance and identify potential issues before they can become costly outages or slowdowns. In this webinar we will explore the new features of the MySQL Enterprise Monitor 2.2 that are designed to save DBAs and Developers time and effort in keeping MySQL systems running at the highest levels of security, performance and availability. Join us and learn about:
Here are the links for the various localized webinars:
English, Italian, German, French

Luca



PlanetMySQL Voting: Vote UP / Vote DOWN";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}s:32:"http://purl.org/dc/elements/1.1/";a:1:{s:7:"creator";a:1:{i:0;a:5:{s:4:"data";s:12:"Luca Olivari";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}}}i:24;a:6:{s:4:"data";s:53:" ";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";s:5:"child";a:3:{s:0:"";a:6:{s:5:"title";a:1:{i:0;a:5:{s:4:"data";s:55:"I'm interviewed today on Radio Vega at 16:00, or not...";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:4:"guid";a:1:{i:0;a:5:{s:4:"data";s:25:"282 at http://openlife.cc";s:7:"attribs";a:1:{s:0:"";a:1:{s:11:"isPermaLink";s:5:"false";}}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:4:"link";a:1:{i:0;a:5:{s:4:"data";s:78:"http://openlife.cc/blogs/2010/june/im-interviewed-today-radio-vega-1600-or-not";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:11:"description";a:1:{i:0;a:5:{s:4:"data";s:465:"I may or may not be interviewed on the Swedish language Radio Vega. The background to this confusing statement is that the Finnish government today publishes a report on the use of open source in Finnish companies and associated risks related to that. I was a contributor to the report since Monty Program is a Finnish open source company. I guess since I speak Swedish, the reporter had scheduled a telephone interview with me after the press conference. read more";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:7:"pubDate";a:1:{i:0;a:5:{s:4:"data";s:31:"Fri, 04 Jun 2010 11:26:38 +0000";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:8:"category";a:3:{i:0;a:5:{s:4:"data";s:5:"MySQL";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:1;a:5:{s:4:"data";s:11:"Open Source";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:2;a:5:{s:4:"data";s:8:"Personal";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}s:40:"http://purl.org/rss/1.0/modules/content/";a:1:{s:7:"encoded";a:1:{i:0;a:5:{s:4:"data";s:901:"

I may or may not be interviewed on the Swedish language Radio Vega. The background to this confusing statement is that the Finnish government today publishes a report on the use of open source in Finnish companies and associated risks related to that. I was a contributor to the report since Monty Program is a Finnish open source company. I guess since I speak Swedish, the reporter had scheduled a telephone interview with me after the press conference.

read more


PlanetMySQL Voting: Vote UP / Vote DOWN";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}s:32:"http://purl.org/dc/elements/1.1/";a:1:{s:7:"creator";a:1:{i:0;a:5:{s:4:"data";s:11:"Henrik Ingo";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}}}i:25;a:6:{s:4:"data";s:68:" ";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";s:5:"child";a:3:{s:0:"";a:6:{s:5:"title";a:1:{i:0;a:5:{s:4:"data";s:76:"mycheckpoint (rev. 132): custom monitoring, custom charts, process list dump";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:4:"guid";a:1:{i:0;a:5:{s:4:"data";s:36:"http://code.openark.org/blog/?p=2463";s:7:"attribs";a:1:{s:0:"";a:1:{s:11:"isPermaLink";s:5:"false";}}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:4:"link";a:1:{i:0;a:5:{s:4:"data";s:105:"http://code.openark.org/blog/mysql/mycheckpoint-rev-132-custom-monitoring-custom-charts-process-list-dump";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:11:"description";a:1:{i:0;a:5:{s:4:"data";s:7754:"Revision 132 of mycheckpoint has been released. New and updated in this revision: Custom monitoring: monitoring & charting for user defined queries HTML reports for custom monitoring Process list dump upon alert notifications Custom monitoring & charts Custom monitoring allows the user to supply with a query, the results of which will be monitored. That is, mycheckpoint monitors the status variables, replication status, OS metrics. But it cannot by itself monitor one’s application. Which is why a user may supply with such query as: SELECT COUNT(*) FROM shopping_cart WHERE is_pending=1 Such a query will tell an online store how many customers are in the midst of shopping. There is no argument that this number is worth monitoring for. Given the above query, mycheckpoint will execute it per sample, and store the query’s result along with all sampled data, to be then aggregated by complex views to answer for: What was the value per given sample? What is the value difference for each sample? What is the change per second, i.e. the rate? mycheckpoint goes one step forward, and explicity records another metric: How much time did it take to take that sample? As another example, a query worth testing for rate: SELECT MAX(shopping_cart_id) FROM shopping_cart The above will provide with the last id. Assuming this is AUTO_INCREMENT, and assuming we’re on auto_increment_increment=1, two samples will allow us to get the number of created carts between those samples. Now, here’s a metric I’d like to read: How many carts are created per second, for each hour of the day? We get all these for free with mycheckpoint, which already does this analysis. All we need to provide is the query, and how we would like it to be visualized (visualization is optional, it is not the only way to diagnose monitored data) graphically: INSERT INTO custom_query (custom_query_id, enabled, query_eval, description, chart_type, chart_order) VALUES (0, 1, 'SELECT COUNT(*) FROM store.shopping_cart WHERE is_pending=1', 'Number of pending carts', 'value', 0); INSERT INTO custom_query (custom_query_id, enabled, query_eval, description, chart_type, chart_order) VALUES (1, 1, 'SELECT MAX(shopping_cart_id) FROM store.shopping_cart', 'Created carts rate', 'value_psec', 0); We can later query for these values, just like we do for normal monitored values: mysql> SELECT id, ts, created_tmp_tables_psec, custom_0, custom_1_psec FROM sv_sample WHERE ts >= NOW() - INTERVAL 1 HOUR; +-------+---------------------+-------------------------+----------+---------------+ | id    | ts                  | created_tmp_tables_psec | custom_0 | custom_1_psec | +-------+---------------------+-------------------------+----------+---------------+ | 50730 | 2010-05-21 19:05:01 |                   16.64 |      448 |          3.02 | | 50731 | 2010-05-21 19:10:02 |                   20.97 |       89 |          1.73 | | 50732 | 2010-05-21 19:15:01 |                   15.70 |      367 |          3.56 | | 50733 | 2010-05-21 19:20:01 |                   18.32 |       54 |          1.43 | | 50734 | 2010-05-21 19:25:01 |                   16.42 |       91 |          1.96 | | 50735 | 2010-05-21 19:30:02 |                   21.93 |      233 |          2.11 | | 50736 | 2010-05-21 19:35:02 |                   14.58 |      176 |          1.91 | | 50737 | 2010-05-21 19:40:01 |                   21.61 |      168 |          1.93 | | 50738 | 2010-05-21 19:45:01 |                   16.05 |      241 |          2.44 | | 50739 | 2010-05-21 19:50:01 |                   19.70 |       46 |          1.19 | | 50740 | 2010-05-21 19:55:01 |                   15.85 |      177 |          2.28 | | 50741 | 2010-05-21 20:00:01 |                   19.04 |        8 |          0.82 | +-------+---------------------+-------------------------+----------+---------------+ Of course, it is also possible to harness mycheckpoint’s views power to generate charts: mysql> SELECT custom_1_psec FROM sv_report_chart_sample\G The rules are: There can (currently) only be 18 custom queries. The custom_query_id must range 0-17 (to be lifted soon). A custom query must return with exactly one row, with exactly one column, which is a kind of integer. Please read my earlier post on custom monitoring to get more background. Custom monitoring HTML reports Custom monitoring comes with a HTML reports, featuring requested charts. See a sample custom report. In this sample report, a few queries are monitored for value (pending rentals, pending downloads) and a few for rates (downloads per second, emails per second etc.). Custom HTML reports come in two flavors: Brief reports, featuring last 24 hours, as in the example above. These are handled by the sv_custom_html_brief view. Full reports, featuring last 24 hours, last 10 days, known history. These take longer to generate, and are handled by the sv_custom_html view. The sample report was generated by issuing: SELECT html FROM sv_custom_html_brief; I won’t go into details here as for how this view generates the HTML code. There is a myriad of view dependencies, with many interesting tricks on the way. But do remember it’s just a view. You don’t need an application (not even mycheckpoint itself) to generate the report. All it takes is a query. Processlist dump When an alert notification fires (an email is prepared to inform on some alert condition), a processlist dump summary is taken and included in email report. It may be useful to understand why the slave is lagging, or exactly why there are so many active threads. The dump summary presents the processlist much as you would see it on SHOW PROCESSLIST, but only lists the active threads, noting down how many sleeping processes there are (PS, thread & process are the same in the terminology of MySQL connections). An example dump looks like this: PROCESSLIST summary: Id: 3 User: system user Host: db: NULL Command: Connect Time: 3168098 State: Waiting for master to send event Info: NULL ------- Id: 4 User: system user Host: db: prod_db Command: Connect Time: 612 State: Updating Info: UPDATE user SET is_offline = 1 WHERE id IN (50440010,50440011) ------- Id: 8916579 User: prod_user Host: localhost db: prod_db Command: Query Time: 1 State: Sending data Info: INSERT IGNORE INTO archive.stat_archive (id, origin, path, ts, content ------- Id: 8916629 User: mycheckpoint Host: localhost db: NULL Command: Query Time: 0 State: NULL Info: SHOW PROCESSLIST ------- Sleeping: 3 processes Future plans Work is going on. These are the non-scheduled future tasks I see: Monitoring InnoDB Plugin & XtraDB status. Interactive charts. See my earlier post. Monitoring for swap activity (Linux only). Enhanced custom queries handling, including auto-deploy upon change of custom queries. A proper man page. Anything else that interests me. Try it out Try out mycheckpoint. It’s a different kind of monitoring solution. You will need basic SQL skills, and in return you’ll get a lot of power under your hands. Download mycheckpoint here Visit the project’s homepage Browse the documentation Report bugs mycheckpoint is released under the New BSD License.";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:7:"pubDate";a:1:{i:0;a:5:{s:4:"data";s:31:"Fri, 04 Jun 2010 09:17:27 +0000";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:8:"category";a:6:{i:0;a:5:{s:4:"data";s:5:"MySQL";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:1;a:5:{s:4:"data";s:6:"Graphs";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:2;a:5:{s:4:"data";s:10:"Monitoring";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:3;a:5:{s:4:"data";s:12:"mycheckpoint";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:4;a:5:{s:4:"data";s:6:"python";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:5;a:5:{s:4:"data";s:7:"scripts";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}s:40:"http://purl.org/rss/1.0/modules/content/";a:1:{s:7:"encoded";a:1:{i:0;a:5:{s:4:"data";s:10379:"

Revision 132 of mycheckpoint has been released. New and updated in this revision:

Custom monitoring & charts

Custom monitoring allows the user to supply with a query, the results of which will be monitored.

That is, mycheckpoint monitors the status variables, replication status, OS metrics. But it cannot by itself monitor one’s application. Which is why a user may supply with such query as:

SELECT COUNT(*) FROM shopping_cart WHERE is_pending=1

Such a query will tell an online store how many customers are in the midst of shopping. There is no argument that this number is worth monitoring for. Given the above query, mycheckpoint will execute it per sample, and store the query’s result along with all sampled data, to be then aggregated by complex views to answer for:

mycheckpoint goes one step forward, and explicity records another metric:

As another example, a query worth testing for rate:

SELECT MAX(shopping_cart_id) FROM shopping_cart

The above will provide with the last id. Assuming this is AUTO_INCREMENT, and assuming we’re on auto_increment_increment=1, two samples will allow us to get the number of created carts between those samples. Now, here’s a metric I’d like to read:

We get all these for free with mycheckpoint, which already does this analysis. All we need to provide is the query, and how we would like it to be visualized (visualization is optional, it is not the only way to diagnose monitored data) graphically:

INSERT INTO
 custom_query (custom_query_id, enabled, query_eval, description, chart_type, chart_order)
 VALUES (0, 1, 'SELECT COUNT(*) FROM store.shopping_cart WHERE is_pending=1', 'Number of pending carts', 'value', 0);

INSERT INTO
 custom_query (custom_query_id, enabled, query_eval, description, chart_type, chart_order)
 VALUES (1, 1, 'SELECT MAX(shopping_cart_id) FROM store.shopping_cart', 'Created carts rate', 'value_psec', 0);

We can later query for these values, just like we do for normal monitored values:

mysql> SELECT id, ts, created_tmp_tables_psec, custom_0, custom_1_psec FROM sv_sample WHERE ts >= NOW() - INTERVAL 1 HOUR;
+-------+---------------------+-------------------------+----------+---------------+
| id    | ts                  | created_tmp_tables_psec | custom_0 | custom_1_psec |
+-------+---------------------+-------------------------+----------+---------------+
| 50730 | 2010-05-21 19:05:01 |                   16.64 |      448 |          3.02 |
| 50731 | 2010-05-21 19:10:02 |                   20.97 |       89 |          1.73 |
| 50732 | 2010-05-21 19:15:01 |                   15.70 |      367 |          3.56 |
| 50733 | 2010-05-21 19:20:01 |                   18.32 |       54 |          1.43 |
| 50734 | 2010-05-21 19:25:01 |                   16.42 |       91 |          1.96 |
| 50735 | 2010-05-21 19:30:02 |                   21.93 |      233 |          2.11 |
| 50736 | 2010-05-21 19:35:02 |                   14.58 |      176 |          1.91 |
| 50737 | 2010-05-21 19:40:01 |                   21.61 |      168 |          1.93 |
| 50738 | 2010-05-21 19:45:01 |                   16.05 |      241 |          2.44 |
| 50739 | 2010-05-21 19:50:01 |                   19.70 |       46 |          1.19 |
| 50740 | 2010-05-21 19:55:01 |                   15.85 |      177 |          2.28 |
| 50741 | 2010-05-21 20:00:01 |                   19.04 |        8 |          0.82 |
+-------+---------------------+-------------------------+----------+---------------+

Of course, it is also possible to harness mycheckpoint’s views power to generate charts:

mysql> SELECT custom_1_psec FROM sv_report_chart_sample\G

The rules are:

Please read my earlier post on custom monitoring to get more background.

Custom monitoring HTML reports

Custom monitoring comes with a HTML reports, featuring requested charts. See a sample custom report.

In this sample report, a few queries are monitored for value (pending rentals, pending downloads) and a few for rates (downloads per second, emails per second etc.).

Custom HTML reports come in two flavors:

The sample report was generated by issuing:

SELECT html FROM sv_custom_html_brief;

I won’t go into details here as for how this view generates the HTML code. There is a myriad of view dependencies, with many interesting tricks on the way. But do remember it’s just a view. You don’t need an application (not even mycheckpoint itself) to generate the report. All it takes is a query.

Processlist dump

When an alert notification fires (an email is prepared to inform on some alert condition), a processlist dump summary is taken and included in email report. It may be useful to understand why the slave is lagging, or exactly why there are so many active threads.

The dump summary presents the processlist much as you would see it on SHOW PROCESSLIST, but only lists the active threads, noting down how many sleeping processes there are (PS, thread & process are the same in the terminology of MySQL connections). An example dump looks like this:

PROCESSLIST summary:

     Id: 3
   User: system user
   Host:
     db: NULL
Command: Connect
   Time: 3168098
  State: Waiting for master to send event
   Info: NULL
-------

     Id: 4
   User: system user
   Host:
     db: prod_db
Command: Connect
   Time: 612
  State: Updating
   Info: UPDATE user SET is_offline = 1 WHERE id IN (50440010,50440011)
-------

     Id: 8916579
   User: prod_user
   Host: localhost
     db: prod_db
Command: Query
   Time: 1
  State: Sending data
   Info: INSERT IGNORE INTO archive.stat_archive (id, origin, path, ts, content
-------

     Id: 8916629
   User: mycheckpoint
   Host: localhost
     db: NULL
Command: Query
   Time: 0
  State: NULL
   Info: SHOW PROCESSLIST
-------
Sleeping: 3 processes

Future plans

Work is going on. These are the non-scheduled future tasks I see:

Try it out

Try out mycheckpoint. It’s a different kind of monitoring solution. You will need basic SQL skills, and in return you’ll get a lot of power under your hands.

mycheckpoint is released under the New BSD License.


PlanetMySQL Voting: Vote UP / Vote DOWN";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}s:32:"http://purl.org/dc/elements/1.1/";a:1:{s:7:"creator";a:1:{i:0;a:5:{s:4:"data";s:12:"Shlomi Noach";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}}}i:26;a:6:{s:4:"data";s:38:" ";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";s:5:"child";a:3:{s:0:"";a:5:{s:5:"title";a:1:{i:0;a:5:{s:4:"data";s:65:"Calpont is pleased to announce high performance sub-query support";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:4:"guid";a:1:{i:0;a:5:{s:4:"data";s:104:"http://infinidb.org/infinidb-blog/calpont-is-pleased-to-announce-high-performance-sub-query-support.html";s:7:"attribs";a:1:{s:0:"";a:1:{s:11:"isPermaLink";s:5:"false";}}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:4:"link";a:1:{i:0;a:5:{s:4:"data";s:104:"http://infinidb.org/infinidb-blog/calpont-is-pleased-to-announce-high-performance-sub-query-support.html";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:11:"description";a:1:{i:0;a:5:{s:4:"data";s:483:" Calpont is pleased to announce high performance sub-query support with the latest alpha version (1.1.2) of InfiniDB.  We’re especially happy about this release because polls done by MySQL have consistently shown – year after year – that slow subquery performance is one of the top items the MySQL community wants fixed in the server. It’s been complex work for sure, but the Calpont engineering team has come through again and delivered highRead More...";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:7:"pubDate";a:1:{i:0;a:5:{s:4:"data";s:31:"Fri, 04 Jun 2010 02:08:54 +0000";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}s:40:"http://purl.org/rss/1.0/modules/content/";a:1:{s:7:"encoded";a:1:{i:0;a:5:{s:4:"data";s:711:"


Calpont is pleased to announce high performance sub-query support with the latest alpha version (1.1.2) of InfiniDB.  We’re especially happy about this release because polls done by MySQL have consistently shown – year after year – that slow subquery performance is one of the top items the MySQL community wants fixed in the server. It’s been complex work for sure, but the Calpont engineering team has come through again and delivered highRead More...
PlanetMySQL Voting: Vote UP / Vote DOWN";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}s:32:"http://purl.org/dc/elements/1.1/";a:1:{s:7:"creator";a:1:{i:0;a:5:{s:4:"data";s:12:"Jim Tommaney";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}}}i:27;a:6:{s:4:"data";s:43:" ";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";s:5:"child";a:3:{s:0:"";a:6:{s:5:"title";a:1:{i:0;a:5:{s:4:"data";s:26:"New feature: Image preview";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:4:"guid";a:1:{i:0;a:5:{s:4:"data";s:48:"http://www.heidisql.com/rss.php?c=1,7&p=5883";s:7:"attribs";a:1:{s:0:"";a:1:{s:11:"isPermaLink";s:5:"false";}}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:4:"link";a:1:{i:0;a:5:{s:4:"data";s:46:"http://www.heidisql.com/forum.php?t=5883#p5883";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:11:"description";a:1:{i:0;a:5:{s:4:"data";s:376:"Users of HeidiSQL 3.x may have missed this feature since grid and BLOB editing was rewritten for v4: A preview area for images. Now it's even more powerful than before: it detects and loads various kinds of images (JPG, PNG, GIF, BMP, PSD, TIF and more). Also, the new preview area does not take valuable space when you're in some grid, as it's placed below the database tree:";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:7:"pubDate";a:1:{i:0;a:5:{s:4:"data";s:31:"Thu, 03 Jun 2010 23:39:01 +0000";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:8:"category";a:1:{i:0;a:5:{s:4:"data";s:4:"News";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}s:40:"http://purl.org/rss/1.0/modules/content/";a:1:{s:7:"encoded";a:1:{i:0;a:5:{s:4:"data";s:687:"Users of HeidiSQL 3.x may have missed this feature since grid and BLOB editing was rewritten for v4: A preview area for images. Now it's even more powerful than before: it detects and loads various kinds of images (JPG, PNG, GIF, BMP, PSD, TIF and more). Also, the new preview area does not take valuable space when you're in some grid, as it's placed below the database tree:



PlanetMySQL Voting: Vote UP / Vote DOWN";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}s:32:"http://purl.org/dc/elements/1.1/";a:1:{s:7:"creator";a:1:{i:0;a:5:{s:4:"data";s:13:"Ansgar Becker";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}}}i:28;a:6:{s:4:"data";s:43:" ";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";s:5:"child";a:3:{s:0:"";a:6:{s:5:"title";a:1:{i:0;a:5:{s:4:"data";s:31:"ON DUPLICATE KEY UPDATE Gotcha!";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:4:"guid";a:1:{i:0;a:5:{s:4:"data";s:70:"tag:blogger.com,1999:blog-8394358003318535041.post-5589915151695230555";s:7:"attribs";a:1:{s:0:"";a:1:{s:11:"isPermaLink";s:5:"false";}}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:4:"link";a:1:{i:0;a:5:{s:4:"data";s:76:"http://mysqlsoapbox.blogspot.com/2010/06/on-duplicate-key-update-gotcha.html";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:11:"description";a:1:{i:0;a:5:{s:4:"data";s:3005:"I know it has been a long time between drinks/posts, but I've been busy -- I promise! :)Today I spent a considerable amount of time trying to figure out why an INSERT SELECT ON DUPLICATE KEY UPDATE was not behaving as I would expect.Here is an example to illustrate:CREATE TABLE t1 (  id INT AUTO_INCREMENT,  num INT NOT NULL DEFAULT 0,  PRIMARY KEY (id));CREATE TABLE t2 (  id INT NOT NULL,  num INT);INSERT INTO t1 VALUES (1, 10);INSERT INTO t2 VALUES (1, NULL);INSERT INTO t1  SELECT id, num  FROM t2ON DUPLICATE KEY UPDATE  num=IFNULL(VALUES(num), t1.num);To convert the above query into plain English -- I'm saying, INSERT into table t1 the id and num fields from the t2 table. If there are already row(s) for any UNIQUE key in the target table, t1, then we should instead UPDATE the existing row. Additionally, we should set the num field to the result of whatever this evaluates to:IFNULL( VALUES(num), t1.num)To explain: VALUES(num) means "The value that is to be placed into the "num" field when it is updated."So we are saying, if a NULL is going to be put into the field "num" then we want to leave the value alone -- set it to "t1.num" -- eg. the value that is already there.One might expect the result of my query to be as follows:testDB:test> SELECT * FROM t1;+----+-----+| id | num |+----+-----+|  1 |  10 |+----+-----+1 row in set (0.00 sec)However, that is not the case -- the actual result is:testDB:test> SELECT * FROM t1;+----+-----+| id | num |+----+-----+|  1 |   0 |+----+-----+1 row in set (0.00 sec)Why is this the case?The hint lies in the result of the INSERT itself:dbp16-int:test> INSERT INTO t1   SELECT id, num   FROM t2 ON DUPLICATE KEY UPDATE   num=IFNULL(VALUES(num), t1.num);Query OK, 0 rows affected, 1 warning (0.00 sec)Records: 1  Duplicates: 0  Warnings: 1Note: 1 warning...testDB:test> SHOW WARNINGS;+---------+------+-----------------------------+| Level   | Code | Message                     |+---------+------+-----------------------------+| Warning | 1048 | Column 'num' cannot be null |+---------+------+-----------------------------+1 row in set (0.00 sec)What is actually happening here is that because the column 'num' in the table t1 is defined as NOT NULL, MySQL is silently converting it to a valid value of 0.So VALUES(num) actually equals 0, thus, it will not evaluate as NULL and the 0 will be INSERTed into the table.This was not my intention and the solution in this case was to allow NULLs on the "num" field of the table t1. It may not always be possible to remove such restrictions if you rely on these automatic conversions by MySQL to "valid values".Something to keep in mind - VALUES() will always evaluate as what would have ended up in the table, which may not necessarily be the same thing as the value that was attempted to be INSERTed.";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:7:"pubDate";a:1:{i:0;a:5:{s:4:"data";s:31:"Thu, 03 Jun 2010 22:58:00 +0000";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:8:"category";a:1:{i:0;a:5:{s:4:"data";s:27:"DUPLICATE KEY UPDATE gotcha";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}s:40:"http://purl.org/rss/1.0/modules/content/";a:1:{s:7:"encoded";a:1:{i:0;a:5:{s:4:"data";s:3947:"I know it has been a long time between drinks/posts, but I've been busy -- I promise! :)

Today I spent a considerable amount of time trying to figure out why an INSERT SELECT ON DUPLICATE KEY UPDATE was not behaving as I would expect.

Here is an example to illustrate:

CREATE TABLE t1 (
  id INT AUTO_INCREMENT,
  num INT NOT NULL DEFAULT 0,
  PRIMARY KEY (id)
);

CREATE TABLE t2 (
  id INT NOT NULL,
  num INT
);

INSERT INTO t1 VALUES (1, 10);
INSERT INTO t2 VALUES (1, NULL);


INSERT INTO t1
  SELECT id, num
  FROM t2
ON DUPLICATE KEY UPDATE
  num=IFNULL(VALUES(num), t1.num);

To convert the above query into plain English -- I'm saying, INSERT into table t1 the id and num fields from the t2 table. If there are already row(s) for any UNIQUE key in the target table, t1, then we should instead UPDATE the existing row. Additionally, we should set the num field to the result of whatever this evaluates to:

IFNULL( VALUES(num), t1.num)

To explain: VALUES(num) means "The value that is to be placed into the "num" field when it is updated."

So we are saying, if a NULL is going to be put into the field "num" then we want to leave the value alone -- set it to "t1.num" -- eg. the value that is already there.

One might expect the result of my query to be as follows:


testDB:test> SELECT * FROM t1;
+----+-----+
| id | num |
+----+-----+
|  1 |  10 |
+----+-----+
1 row in set (0.00 sec)



However, that is not the case -- the actual result is:



testDB:test> SELECT * FROM t1;
+----+-----+
| id | num |
+----+-----+
|  1 |   0 |
+----+-----+
1 row in set (0.00 sec)


Why is this the case?

The hint lies in the result of the INSERT itself:


dbp16-int:test> INSERT INTO t1   SELECT id, num   FROM t2 ON DUPLICATE KEY UPDATE   num=IFNULL(VALUES(num), t1.num);
Query OK, 0 rows affected, 1 warning (0.00 sec)
Records: 1  Duplicates: 0  Warnings: 1

Note: 1 warning...

testDB:test> SHOW WARNINGS;
+---------+------+-----------------------------+
| Level   | Code | Message                     |
+---------+------+-----------------------------+
| Warning | 1048 | Column 'num' cannot be null |
+---------+------+-----------------------------+
1 row in set (0.00 sec)

What is actually happening here is that because the column 'num' in the table t1 is defined as NOT NULL, MySQL is silently converting it to a valid value of 0.

So VALUES(num) actually equals 0, thus, it will not evaluate as NULL and the 0 will be INSERTed into the table.

This was not my intention and the solution in this case was to allow NULLs on the "num" field of the table t1. It may not always be possible to remove such restrictions if you rely on these automatic conversions by MySQL to "valid values".

Something to keep in mind - VALUES() will always evaluate as what would have ended up in the table, which may not necessarily be the same thing as the value that was attempted to be INSERTed.

PlanetMySQL Voting: Vote UP / Vote DOWN";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}s:32:"http://purl.org/dc/elements/1.1/";a:1:{s:7:"creator";a:1:{i:0;a:5:{s:4:"data";s:15:"Lachlan Mulcahy";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}}}i:29;a:6:{s:4:"data";s:58:" ";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";s:5:"child";a:3:{s:0:"";a:6:{s:5:"title";a:1:{i:0;a:5:{s:4:"data";s:37:"When SET GLOBAL affects SESSION scope";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:4:"guid";a:1:{i:0;a:5:{s:4:"data";s:38:"http://ronaldbradford.com/blog/?p=2849";s:7:"attribs";a:1:{s:0:"";a:1:{s:11:"isPermaLink";s:5:"false";}}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:4:"link";a:1:{i:0;a:5:{s:4:"data";s:80:"http://ronaldbradford.com/blog/when-set-global-affects-session-scope-2010-06-03/";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:11:"description";a:1:{i:0;a:5:{s:4:"data";s:3297:"We have all been caught out with using SET and not realizing that the default GLOBAL Scope (since 5.0.2) does not change the current SESSION scope. I was not aware until today that changing GLOBAL scope has some exceptions that also automatically affect SESSION scope. What I expected with a change in the GLOBAL scope is no affect SESSION scope. For example. mysql> SHOW GLOBAL VARIABLES LIKE 'read_buffer_size'; +------------------+--------+ | Variable_name | Value | +------------------+--------+ | read_buffer_size | 131072 | +------------------+--------+ 1 row in set (0.00 sec) mysql> SHOW SESSION VARIABLES LIKE 'read_buffer_size'; +------------------+--------+ | Variable_name | Value | +------------------+--------+ | read_buffer_size | 131072 | +------------------+--------+ 1 row in set (0.00 sec) mysql> SET GLOBAL read_buffer_size=1024*256; Query OK, 0 rows affected (0.00 sec) mysql> SHOW GLOBAL VARIABLES LIKE 'read_buffer_size'; +------------------+--------+ | Variable_name | Value | +------------------+--------+ | read_buffer_size | 262144 | +------------------+--------+ 1 row in set (0.00 sec) mysql> SHOW SESSION VARIABLES LIKE 'read_buffer_size'; +------------------+--------+ | Variable_name | Value | +------------------+--------+ | read_buffer_size | 131072 | +------------------+--------+ 1 row in set (0.00 sec) However I was no prepared for this when changing an important variable for transaction management. mysql> SHOW GLOBAL VARIABLES LIKE 'autocommit'; +---------------+-------+ | Variable_name | Value | +---------------+-------+ | autocommit | ON | +---------------+-------+ 1 row in set (0.00 sec) mysql> SHOW SESSION VARIABLES LIKE 'autocommit'; +---------------+-------+ | Variable_name | Value | +---------------+-------+ | autocommit | ON | +---------------+-------+ 1 row in set (0.00 sec) mysql> SET autocommit=0; Query OK, 0 rows affected (0.00 sec) mysql> SHOW GLOBAL VARIABLES LIKE 'autocommit'; +---------------+-------+ | Variable_name | Value | +---------------+-------+ | autocommit | OFF | +---------------+-------+ 1 row in set (0.00 sec) mysql> SHOW SESSION VARIABLES LIKE 'autocommit'; +---------------+-------+ | Variable_name | Value | +---------------+-------+ | autocommit | OFF | +---------------+-------+ 1 row in set (0.00 sec) However even more perplexing was the following message: mysql> SET GLOBAL autocommit=0; ERROR 1228 (HY000): Variable 'autocommit' is a SESSION variable and can't be used with SET GLOBAL So this is another case were the definition of variables is not applicable in a GLOBAL level, yet the tools of the trade represent in some manner misleading information. To prove my point, here is another new concurrent session started after the above. mysql> SHOW GLOBAL VARIABLES LIKE 'autocommit'; +---------------+-------+ | Variable_name | Value | +---------------+-------+ | autocommit | ON | +---------------+-------+ 1 row in set (0.01 sec) mysql> SHOW SESSION VARIABLES LIKE 'autocommit'; +---------------+-------+ | Variable_name | Value | +---------------+-------+ | autocommit | ON | +---------------+-------+ 1 row in set (0.00 sec) The MySQL Documentation also had an incorrect specification with description regarding this scope, Bug #54215";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:7:"pubDate";a:1:{i:0;a:5:{s:4:"data";s:31:"Thu, 03 Jun 2010 18:43:24 +0000";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:8:"category";a:4:{i:0;a:5:{s:4:"data";s:9:"Databases";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:1;a:5:{s:4:"data";s:5:"MySQL";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:2;a:5:{s:4:"data";s:12:"Professional";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:3;a:5:{s:4:"data";s:14:"variable scope";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}s:40:"http://purl.org/rss/1.0/modules/content/";a:1:{s:7:"encoded";a:1:{i:0;a:5:{s:4:"data";s:3761:"

We have all been caught out with using SET and not realizing that the default GLOBAL Scope (since 5.0.2) does not change the current SESSION scope.

I was not aware until today that changing GLOBAL scope has some exceptions that also automatically affect SESSION scope.

What I expected with a change in the GLOBAL scope is no affect SESSION scope. For example.

mysql> SHOW GLOBAL VARIABLES LIKE 'read_buffer_size';
+------------------+--------+
| Variable_name    | Value  |
+------------------+--------+
| read_buffer_size | 131072 |
+------------------+--------+
1 row in set (0.00 sec)

mysql> SHOW SESSION VARIABLES LIKE 'read_buffer_size';
+------------------+--------+
| Variable_name    | Value  |
+------------------+--------+
| read_buffer_size | 131072 |
+------------------+--------+
1 row in set (0.00 sec)

mysql> SET GLOBAL read_buffer_size=1024*256;
Query OK, 0 rows affected (0.00 sec)

mysql> SHOW GLOBAL VARIABLES LIKE 'read_buffer_size';
+------------------+--------+
| Variable_name    | Value  |
+------------------+--------+
| read_buffer_size | 262144 |
+------------------+--------+
1 row in set (0.00 sec)

mysql> SHOW SESSION VARIABLES LIKE 'read_buffer_size';
+------------------+--------+
| Variable_name    | Value  |
+------------------+--------+
| read_buffer_size | 131072 |
+------------------+--------+
1 row in set (0.00 sec)

However I was no prepared for this when changing an important variable for transaction management.

mysql> SHOW GLOBAL VARIABLES LIKE 'autocommit';
+---------------+-------+
| Variable_name | Value |
+---------------+-------+
| autocommit    | ON    |
+---------------+-------+
1 row in set (0.00 sec)

mysql> SHOW SESSION VARIABLES LIKE 'autocommit';
+---------------+-------+
| Variable_name | Value |
+---------------+-------+
| autocommit    | ON    |
+---------------+-------+
1 row in set (0.00 sec)

mysql> SET autocommit=0;
Query OK, 0 rows affected (0.00 sec)

mysql> SHOW GLOBAL VARIABLES LIKE 'autocommit';
+---------------+-------+
| Variable_name | Value |
+---------------+-------+
| autocommit    | OFF   |
+---------------+-------+
1 row in set (0.00 sec)

mysql> SHOW SESSION VARIABLES LIKE 'autocommit';
+---------------+-------+
| Variable_name | Value |
+---------------+-------+
| autocommit    | OFF   |
+---------------+-------+
1 row in set (0.00 sec)

However even more perplexing was the following message:

mysql> SET GLOBAL autocommit=0;
ERROR 1228 (HY000): Variable 'autocommit' is a SESSION variable and can't be used with SET GLOBAL

So this is another case were the definition of variables is not applicable in a GLOBAL level, yet the tools of the trade represent in some manner misleading information.
To prove my point, here is another new concurrent session started after the above.

mysql> SHOW GLOBAL VARIABLES LIKE 'autocommit';
+---------------+-------+
| Variable_name | Value |
+---------------+-------+
| autocommit    | ON    |
+---------------+-------+
1 row in set (0.01 sec)

mysql> SHOW SESSION VARIABLES LIKE 'autocommit';
+---------------+-------+
| Variable_name | Value |
+---------------+-------+
| autocommit    | ON    |
+---------------+-------+
1 row in set (0.00 sec)

The MySQL Documentation also had an incorrect specification with description regarding this scope, Bug #54215


PlanetMySQL Voting: Vote UP / Vote DOWN";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}s:32:"http://purl.org/dc/elements/1.1/";a:1:{s:7:"creator";a:1:{i:0;a:5:{s:4:"data";s:15:"Ronald Bradford";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}}}i:30;a:6:{s:4:"data";s:68:" ";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";s:5:"child";a:3:{s:0:"";a:6:{s:5:"title";a:1:{i:0;a:5:{s:4:"data";s:40:"Best Practices: Additional User Security";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:4:"guid";a:1:{i:0;a:5:{s:4:"data";s:38:"http://ronaldbradford.com/blog/?p=2842";s:7:"attribs";a:1:{s:0:"";a:1:{s:11:"isPermaLink";s:5:"false";}}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:4:"link";a:1:{i:0;a:5:{s:4:"data";s:82:"http://ronaldbradford.com/blog/best-practices-additional-user-security-2010-06-03/";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:11:"description";a:1:{i:0;a:5:{s:4:"data";s:2395:"By default MySQL allows you to create user accounts and privileges with no password. In my earlier MySQL Best Practices: User Security I describe how to address the default installation empty passwords. For new user accounts, you can improve this default behavior using the SQL_MODE variable, with a value of NO_AUTO_CREATE_USER. As detailed via the 5.1 Reference Manual NO_AUTO_CREATE_USER Prevent the GRANT statement from automatically creating new users if it would otherwise do so, unless a nonempty password also is specified. Having set this variable I attempted to show the error of operation to demonstrate in my upcoming “MySQL Idiosyncrasies that bite” presentation. Confirm Settings mysql> show global variables like 'sql_mode'; +---------------+---------------------+ | Variable_name | Value | +---------------+---------------------+ | sql_mode | NO_AUTO_CREATE_USER | +---------------+---------------------+ 1 row in set (0.00 sec) mysql> show session variables like 'sql_mode'; +---------------+---------------------+ | Variable_name | Value | +---------------+---------------------+ | sql_mode | NO_AUTO_CREATE_USER | +---------------+---------------------+ 1 row in set (0.00 sec) Create error condition mysql> CREATE USER superuser@localhost; Query OK, 0 rows affected (0.00 sec) mysql> GRANT ALL ON *.* TO superuser@localhost; Query OK, 0 rows affected (0.00 sec) mysql> exit What the? Surely this isn’t right. $ mysql -usuperuser mysql> SHOW GRANTS; +--------------------------------------------------------+ | Grants for superuser@localhost | +--------------------------------------------------------+ | GRANT ALL PRIVILEGES ON *.* TO 'superuser'@'localhost' | +--------------------------------------------------------+ mysql> SELECT VERSION(); +-----------+ | VERSION() | +-----------+ | 5.1.39 | +-----------+ Well that’s broken functionality. What should happen as described in Bug #43938 is a cryptic message as reproduced below. mysql> GRANT SELECT ON foo.* TO 'geert12'@'localhost'; ERROR 1133 (42000): Can't find any matching row in the user table mysql> GRANT SELECT ON *.* TO geert12@localhost IDENTIFIED BY 'foobar'; Query OK, 0 rows affected (0.00 sec) It seems however that the user of CREATE USER first nullifies this expected behavior, hence new Bug #54208.";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:7:"pubDate";a:1:{i:0;a:5:{s:4:"data";s:31:"Thu, 03 Jun 2010 16:12:50 +0000";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:8:"category";a:6:{i:0;a:5:{s:4:"data";s:9:"Databases";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:1;a:5:{s:4:"data";s:5:"MySQL";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:2;a:5:{s:4:"data";s:12:"Professional";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:3;a:5:{s:4:"data";s:14:"best practices";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:4;a:5:{s:4:"data";s:3:"bug";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:5;a:5:{s:4:"data";s:8:"security";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}s:40:"http://purl.org/rss/1.0/modules/content/";a:1:{s:7:"encoded";a:1:{i:0;a:5:{s:4:"data";s:3051:"

By default MySQL allows you to create user accounts and privileges with no password. In my earlier MySQL Best Practices: User Security I describe how to address the default installation empty passwords.

For new user accounts, you can improve this default behavior using the SQL_MODE variable, with a value of NO_AUTO_CREATE_USER. As detailed via the 5.1 Reference Manual

NO_AUTO_CREATE_USER

Prevent the GRANT statement from automatically creating new users if it would otherwise do so, unless a nonempty password also is specified.

Having set this variable I attempted to show the error of operation to demonstrate in my upcoming “MySQL Idiosyncrasies that bite” presentation.

Confirm Settings

mysql> show global variables like 'sql_mode';
+---------------+---------------------+
| Variable_name | Value               |
+---------------+---------------------+
| sql_mode      | NO_AUTO_CREATE_USER |
+---------------+---------------------+
1 row in set (0.00 sec)

mysql> show session variables like 'sql_mode';
+---------------+---------------------+
| Variable_name | Value               |
+---------------+---------------------+
| sql_mode      | NO_AUTO_CREATE_USER |
+---------------+---------------------+
1 row in set (0.00 sec)

Create error condition

mysql> CREATE USER superuser@localhost;
Query OK, 0 rows affected (0.00 sec)
mysql> GRANT ALL ON *.* TO superuser@localhost;
Query OK, 0 rows affected (0.00 sec)
mysql> exit

What the? Surely this isn’t right.

$ mysql -usuperuser

mysql> SHOW GRANTS;
+--------------------------------------------------------+
| Grants for superuser@localhost                         |
+--------------------------------------------------------+
| GRANT ALL PRIVILEGES ON *.* TO 'superuser'@'localhost' |
+--------------------------------------------------------+

mysql> SELECT VERSION();
+-----------+
| VERSION() |
+-----------+
| 5.1.39    |
+-----------+

Well that’s broken functionality.

What should happen as described in Bug #43938 is a cryptic message as reproduced below.

mysql> GRANT SELECT ON foo.* TO 'geert12'@'localhost';
ERROR 1133 (42000): Can't find any matching row in the user table
mysql> GRANT SELECT ON *.* TO geert12@localhost IDENTIFIED BY 'foobar';
Query OK, 0 rows affected (0.00 sec)

It seems however that the user of CREATE USER first nullifies this expected behavior, hence new Bug #54208.


PlanetMySQL Voting: Vote UP / Vote DOWN";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}s:32:"http://purl.org/dc/elements/1.1/";a:1:{s:7:"creator";a:1:{i:0;a:5:{s:4:"data";s:15:"Ronald Bradford";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}}}i:31;a:6:{s:4:"data";s:88:" ";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";s:5:"child";a:3:{s:0:"";a:6:{s:5:"title";a:1:{i:0;a:5:{s:4:"data";s:21:"Berkeley DB 5 and PHP";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:4:"guid";a:1:{i:0;a:5:{s:4:"data";s:48:"http://schlueters.de/blog/archives/134-guid.html";s:7:"attribs";a:1:{s:0:"";a:1:{s:11:"isPermaLink";s:5:"false";}}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:4:"link";a:1:{i:0;a:5:{s:4:"data";s:65:"http://schlueters.de/blog/archives/134-Berkeley-DB-5-and-PHP.html";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:11:"description";a:1:{i:0;a:5:{s:4:"data";s:16221:"Everybody working on Unix or in the database world stumbles over Oracle Berkeley DB every now and then. DB is an Open Source embedded database used by applications like OpenLDAP or Postfix. Traditionally it followed mostly a key-value access pattern. Now what caught my attention was the fact that the recently released DB 5.0 provides an SQLite-like C API with the promise of providing better concurrency and performance than regular SQLite. Time to give it a shot. So I grabbed the source distribution, checked the documentation and saw that I shall use the --enable-sql_compat configure option so DB creates a libsqlite3.so which can be used by PHP's build system. On my system compiling this worked like a charm and after linking PHP against it the SQLite3 extension as well as th SQLite PDO driver worked as before. I didn't do any benchmarks, yet, but I would be interested to read some results. Oh and yes, according to the documentation there are a few minor changes in behavior, this should only affect very few PHP users though. So the whole process after extracting DB and PHP sources looked like this: $ cd db-5.0.21/build_unix $ ../dist/configure --prefix=/opt/db-5.0.21 --enable-sql_compat --enable-cxx $ make # make install $ cd ../.. $ cd php-src $ ./configure --with-sqlite3=/opt/db-5.0.21 --with-pdo-sqlite=/opt/db-5.0.21 --with-other-php-options $ make # make install Now you don't have to use the SQLite interface for using DB but can also use PHP's dba extension. In current releases of PHP PHP expects at most DB version number 4 for the DBA driver, but a patch was committed to PHP's svn repository so upcoming PHP releases can be build using the latest DB version by adding --with-db4=/opt/db-5.0.21 to PHP's configure line. Now I can hear you complain: "three ways to access a database isn't enough!" - which is true as the SQLite3 extension as well as the PDO SQLite driver are limited to SQLite capabilities and DBA, as an abstraction layer for Berkeley DB-like databases, only offers some basic operations, too. But there's hope: While browsing through the DB source distribution I stumbled over a directory called php_db4 which obviously caught my further attention, so the DB developers are bundling a DB-specific PHP extension wrapping DB's native C++ API in an PHP OO API. To try it simply follow the steps from the PHP manual for compiling custom extensions: $ cd db-5.0.21/php_db4 $ phpize $ ./configure --with-db4=/opt/db-5.0.21 $ make # make install And then a quick check using reflection to see what functionality is provided by this extension: $ php -dextension=db4.so --re db4 Extension [ extension #45 db4 version 0.9 ] { - Constants [210] { Constant [ integer DB_VERSION_MAJOR ] { 5 } Constant [ integer DB_VERSION_MINOR ] { 0 } Constant [ integer DB_VERSION_PATCH ] { 21 } Constant [ string DB_VERSION_STRING ] { Berkeley DB 5.0.21: (March 30, 2010) } Constant [ integer DB_MAX_PAGES ] { 4294967295 } Constant [ integer DB_MAX_RECORDS ] { 4294967295 } Constant [ integer DB_DBT_APPMALLOC ] { 1 } Constant [ integer DB_DBT_ISSET ] { 8 } Constant [ integer DB_DBT_MALLOC ] { 16 } Constant [ integer DB_DBT_PARTIAL ] { 64 } Constant [ integer DB_DBT_REALLOC ] { 128 } Constant [ integer DB_DBT_USERMEM ] { 1024 } Constant [ integer DB_DBT_DUPOK ] { 4 } Constant [ integer DB_CREATE ] { 1 } Constant [ integer DB_CXX_NO_EXCEPTIONS ] { 2 } Constant [ integer DB_FORCE ] { 1 } Constant [ integer DB_NOMMAP ] { 8 } Constant [ integer DB_RDONLY ] { 1024 } Constant [ integer DB_RECOVER ] { 2 } Constant [ integer DB_MULTIVERSION ] { 4 } Constant [ integer DB_TXN_SNAPSHOT ] { 2 } Constant [ integer DB_THREAD ] { 16 } Constant [ integer DB_TRUNCATE ] { 32768 } Constant [ integer DB_TXN_NOSYNC ] { 1 } Constant [ integer DB_TXN_NOT_DURABLE ] { 2 } Constant [ integer DB_USE_ENVIRON ] { 4 } Constant [ integer DB_USE_ENVIRON_ROOT ] { 8 } Constant [ integer DB_AUTO_COMMIT ] { 256 } Constant [ integer DB_DIRTY_READ ] { 512 } Constant [ integer DB_DEGREE_2 ] { 1024 } Constant [ integer DB_READ_COMMITTED ] { 1024 } Constant [ integer DB_READ_UNCOMMITTED ] { 512 } Constant [ integer DB_NO_AUTO_COMMIT ] { 8192 } Constant [ integer DB_RPCCLIENT ] { 1 } Constant [ integer DB_INIT_CDB ] { 64 } Constant [ integer DB_INIT_LOCK ] { 128 } Constant [ integer DB_INIT_LOG ] { 256 } Constant [ integer DB_INIT_MPOOL ] { 512 } Constant [ integer DB_INIT_REP ] { 1024 } Constant [ integer DB_INIT_TXN ] { 2048 } Constant [ integer DB_JOINENV ] { 0 } Constant [ integer DB_LOCKDOWN ] { 4096 } Constant [ integer DB_PRIVATE ] { 8192 } Constant [ integer DB_RECOVER_FATAL ] { 16384 } Constant [ integer DB_SYSTEM_MEM ] { 65536 } Constant [ integer DB_EXCL ] { 64 } Constant [ integer DB_FCNTL_LOCKING ] { 2048 } Constant [ integer DB_RDWRMASTER ] { 16384 } Constant [ integer DB_WRITEOPEN ] { 65536 } Constant [ integer DB_TXN_NOWAIT ] { 16 } Constant [ integer DB_TXN_SYNC ] { 4 } Constant [ integer DB_ENCRYPT_AES ] { 1 } Constant [ integer DB_CDB_ALLDB ] { 64 } Constant [ integer DB_DIRECT_DB ] { 128 } Constant [ integer DB_NOLOCKING ] { 1024 } Constant [ integer DB_NOPANIC ] { 2048 } Constant [ integer DB_OVERWRITE ] { 4096 } Constant [ integer DB_PANIC_ENVIRONMENT ] { 8192 } Constant [ integer DB_REGION_INIT ] { 16384 } Constant [ integer DB_TIME_NOTGRANTED ] { 32768 } Constant [ integer DB_TXN_WRITE_NOSYNC ] { 32 } Constant [ integer DB_YIELDCPU ] { 65536 } Constant [ integer DB_UPGRADE ] { 1 } Constant [ integer DB_VERIFY ] { 2 } Constant [ integer DB_DIRECT ] { 16 } Constant [ integer DB_EXTENT ] { 64 } Constant [ integer DB_ODDFILESIZE ] { 128 } Constant [ integer DB_CHKSUM ] { 8 } Constant [ integer DB_DUP ] { 16 } Constant [ integer DB_DUPSORT ] { 4 } Constant [ integer DB_ENCRYPT ] { 1 } Constant [ integer DB_RECNUM ] { 64 } Constant [ integer DB_RENUMBER ] { 128 } Constant [ integer DB_REVSPLITOFF ] { 256 } Constant [ integer DB_SNAPSHOT ] { 512 } Constant [ integer DB_STAT_CLEAR ] { 1 } Constant [ integer DB_JOIN_NOSORT ] { 1 } Constant [ integer DB_AGGRESSIVE ] { 1 } Constant [ integer DB_NOORDERCHK ] { 2 } Constant [ integer DB_ORDERCHKONLY ] { 4 } Constant [ integer DB_PR_PAGE ] { 16 } Constant [ integer DB_PR_RECOVERYTEST ] { 32 } Constant [ integer DB_PRINTABLE ] { 8 } Constant [ integer DB_SALVAGE ] { 64 } Constant [ integer DB_REP_NOBUFFER ] { 2 } Constant [ integer DB_REP_PERMANENT ] { 4 } Constant [ integer DB_LOCKVERSION ] { 1 } Constant [ integer DB_FILE_ID_LEN ] { 20 } Constant [ integer DB_LOCK_NORUN ] { 0 } Constant [ integer DB_LOCK_DEFAULT ] { 1 } Constant [ integer DB_LOCK_EXPIRE ] { 2 } Constant [ integer DB_LOCK_MAXLOCKS ] { 3 } Constant [ integer DB_LOCK_MINLOCKS ] { 5 } Constant [ integer DB_LOCK_MINWRITE ] { 6 } Constant [ integer DB_LOCK_OLDEST ] { 7 } Constant [ integer DB_LOCK_RANDOM ] { 8 } Constant [ integer DB_LOCK_YOUNGEST ] { 9 } Constant [ integer DB_LOCK_NOWAIT ] { 1 } Constant [ integer DB_LOCK_RECORD ] { 2 } Constant [ integer DB_LOCK_SET_TIMEOUT ] { 4 } Constant [ integer DB_LOCK_SWITCH ] { 8 } Constant [ integer DB_LOCK_UPGRADE ] { 16 } Constant [ integer DB_HANDLE_LOCK ] { 1 } Constant [ integer DB_RECORD_LOCK ] { 2 } Constant [ integer DB_PAGE_LOCK ] { 3 } Constant [ integer DB_LOGVERSION ] { 17 } Constant [ integer DB_LOGOLDVER ] { 8 } Constant [ integer DB_LOGMAGIC ] { 264584 } Constant [ integer DB_ARCH_ABS ] { 1 } Constant [ integer DB_ARCH_DATA ] { 2 } Constant [ integer DB_ARCH_LOG ] { 4 } Constant [ integer DB_ARCH_REMOVE ] { 8 } Constant [ integer DB_FLUSH ] { 1 } Constant [ integer DB_LOG_CHKPNT ] { 2 } Constant [ integer DB_LOG_COMMIT ] { 4 } Constant [ integer DB_LOG_NOCOPY ] { 8 } Constant [ integer DB_LOG_NOT_DURABLE ] { 16 } Constant [ integer DB_LOG_WRNOSYNC ] { 32 } Constant [ integer DB_user_BEGIN ] { 10000 } Constant [ integer DB_debug_FLAG ] { 2147483648 } Constant [ integer DB_LOG_DISK ] { 1 } Constant [ integer DB_LOG_LOCKED ] { 2 } Constant [ integer DB_LOG_SILENT_ERR ] { 4 } Constant [ integer DB_MPOOL_CREATE ] { 1 } Constant [ integer DB_MPOOL_LAST ] { 16 } Constant [ integer DB_MPOOL_NEW ] { 32 } Constant [ integer DB_MPOOL_DIRTY ] { 2 } Constant [ integer DB_MPOOL_DISCARD ] { 1 } Constant [ integer DB_MPOOL_NOFILE ] { 1 } Constant [ integer DB_MPOOL_UNLINK ] { 2 } Constant [ integer DB_TXNVERSION ] { 1 } Constant [ integer DB_GID_SIZE ] { 128 } Constant [ integer DB_EID_BROADCAST ] { -1 } Constant [ integer DB_EID_INVALID ] { -2 } Constant [ integer DB_REP_CLIENT ] { 1 } Constant [ integer DB_REP_MASTER ] { 2 } Constant [ integer DB_RENAMEMAGIC ] { 198656 } Constant [ integer DB_BTREEVERSION ] { 9 } Constant [ integer DB_BTREEOLDVER ] { 8 } Constant [ integer DB_BTREEMAGIC ] { 340322 } Constant [ integer DB_HASHVERSION ] { 9 } Constant [ integer DB_HASHOLDVER ] { 7 } Constant [ integer DB_HASHMAGIC ] { 398689 } Constant [ integer DB_QAMVERSION ] { 4 } Constant [ integer DB_QAMOLDVER ] { 3 } Constant [ integer DB_QAMMAGIC ] { 270931 } Constant [ integer DB_AFTER ] { 1 } Constant [ integer DB_APPEND ] { 2 } Constant [ integer DB_BEFORE ] { 3 } Constant [ integer DB_CONSUME ] { 4 } Constant [ integer DB_CONSUME_WAIT ] { 5 } Constant [ integer DB_CURRENT ] { 6 } Constant [ integer DB_FAST_STAT ] { 1 } Constant [ integer DB_FIRST ] { 7 } Constant [ integer DB_GET_BOTH ] { 8 } Constant [ integer DB_GET_BOTHC ] { 9 } Constant [ integer DB_GET_BOTH_RANGE ] { 10 } Constant [ integer DB_GET_RECNO ] { 11 } Constant [ integer DB_JOIN_ITEM ] { 12 } Constant [ integer DB_KEYFIRST ] { 13 } Constant [ integer DB_KEYLAST ] { 14 } Constant [ integer DB_LAST ] { 15 } Constant [ integer DB_NEXT ] { 16 } Constant [ integer DB_NEXT_DUP ] { 17 } Constant [ integer DB_NEXT_NODUP ] { 18 } Constant [ integer DB_NODUPDATA ] { 19 } Constant [ integer DB_NOOVERWRITE ] { 20 } Constant [ integer DB_NOSYNC ] { 21 } Constant [ integer DB_POSITION ] { 23 } Constant [ integer DB_PREV ] { 24 } Constant [ integer DB_PREV_NODUP ] { 26 } Constant [ integer DB_SET ] { 27 } Constant [ integer DB_SET_LOCK_TIMEOUT ] { 1 } Constant [ integer DB_SET_RANGE ] { 28 } Constant [ integer DB_SET_RECNO ] { 29 } Constant [ integer DB_SET_TXN_NOW ] { 8 } Constant [ integer DB_SET_TXN_TIMEOUT ] { 2 } Constant [ integer DB_UPDATE_SECONDARY ] { 30 } Constant [ integer DB_WRITECURSOR ] { 8 } Constant [ integer DB_WRITELOCK ] { 16 } Constant [ integer DB_OPFLAGS_MASK ] { 255 } Constant [ integer DB_MULTIPLE ] { 2048 } Constant [ integer DB_MULTIPLE_KEY ] { 16384 } Constant [ integer DB_RMW ] { 8192 } Constant [ integer DB_DONOTINDEX ] { -30998 } Constant [ integer DB_KEYEMPTY ] { -30996 } Constant [ integer DB_KEYEXIST ] { -30995 } Constant [ integer DB_LOCK_DEADLOCK ] { -30994 } Constant [ integer DB_LOCK_NOTGRANTED ] { -30993 } Constant [ integer DB_NOSERVER ] { -30990 } Constant [ integer DB_NOSERVER_HOME ] { -30989 } Constant [ integer DB_NOSERVER_ID ] { -30988 } Constant [ integer DB_NOTFOUND ] { -30987 } Constant [ integer DB_OLD_VERSION ] { -30986 } Constant [ integer DB_PAGE_NOTFOUND ] { -30985 } Constant [ integer DB_REP_DUPMASTER ] { -30984 } Constant [ integer DB_REP_HANDLE_DEAD ] { -30983 } Constant [ integer DB_REP_HOLDELECTION ] { -30982 } Constant [ integer DB_REP_ISPERM ] { -30980 } Constant [ integer DB_REP_NEWMASTER ] { -30893 } Constant [ integer DB_REP_NEWSITE ] { -30976 } Constant [ integer DB_REP_NOTPERM ] { -30975 } Constant [ integer DB_REP_UNAVAIL ] { -30974 } Constant [ integer DB_RUNRECOVERY ] { -30973 } Constant [ integer DB_SECONDARY_BAD ] { -30972 } Constant [ integer DB_VERIFY_BAD ] { -30970 } Constant [ integer DB_VERB_DEADLOCK ] { 1 } Constant [ integer DB_VERB_RECOVERY ] { 8 } Constant [ integer DB_VERB_REPLICATION ] { 32 } Constant [ integer DB_VERB_WAITSFOR ] { 32768 } } - Classes [4] { Class [ class db4txn ] { - Constants [0] { } - Static properties [0] { } - Static methods [0] { } - Properties [0] { } - Methods [8] { Method [ public method abort ] { } Method [ public method commit ] { } Method [ public method discard ] { } Method [ public method id ] { } Method [ public method set_timeout ] { } Method [ public method set_name ] { } Method [ public method get_name ] { } Method [ public method db4txn ] { } } } Class [ class db4cursor ] { - Constants [0] { } - Static properties [0] { } - Static methods [0] { } - Properties [0] { } - Methods [7] { Method [ public method close ] { } Method [ public method count ] { } Method [ public method del ] { } Method [ public method dup ] { } Method [ public method get ] { - Parameters [2] { Parameter #0 [ &$param0 ] Parameter #1 [ &$param1 ] } } Method [ public method put ] { } Method [ public method pget ] { - Parameters [2] { Parameter #0 [ &$param0 ] Parameter #1 [ &$param1 ] } } } } Class [ class db4 ] { - Constants [0] { } - Static properties [0] { } - Static methods [0] { } - Properties [0] { } - Methods [15] { Method [ public method db4 ] { } Method [ public method open ] { } Method [ public method close ] { } Method [ public method cursor ] { } Method [ public method del ] { } Method [ public method get ] { } Method [ public method get_encrypt_flags ] { } Method [ public method pget ] { } Method [ public method get_type ] { } Method [ public method join ] { } Method [ public method put ] { } Method [ public method set_encrypt ] { } Method [ public method stat ] { } Method [ public method sync ] { } Method [ public method truncate ] { } } } Class [ class db4env ] { - Constants [0] { } - Static properties [0] { } - Static methods [0] { } - Properties [0] { } - Methods [11] { Method [ public method db4env ] { } Method [ public method close ] { } Method [ public method dbremove ] { } Method [ public method dbrename ] { } Method [ public method get_encrypt ] { } Method [ public method open ] { } Method [ public method remove ] { } Method [ public method set_data_dir ] { } Method [ public method set_encrypt ] { } Method [ public method txn_begin ] { } Method [ public method txn_checkpoint ] { } } } } } The extension bundles some examples to get you started with Berkeley DB. Even though I'd prefer a relational client-server database for most use cases I can image situations where a simple embedded solutions has its benefits, especially if combined with other Oracle Berkeley DB based applications. ";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:7:"pubDate";a:1:{i:0;a:5:{s:4:"data";s:31:"Thu, 03 Jun 2010 13:54:00 +0000";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:8:"category";a:10:{i:0;a:5:{s:4:"data";s:3:"PHP";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:1;a:5:{s:4:"data";s:11:"berkeley db";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:2;a:5:{s:4:"data";s:6:"coding";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:3;a:5:{s:4:"data";s:8:"database";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:4;a:5:{s:4:"data";s:2:"db";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:5;a:5:{s:4:"data";s:6:"oracle";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:6;a:5:{s:4:"data";s:3:"php";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:7;a:5:{s:4:"data";s:14:"php extensions";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:8;a:5:{s:4:"data";s:6:"sqlite";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:9;a:5:{s:4:"data";s:7:"sqlite3";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}s:40:"http://purl.org/rss/1.0/modules/content/";a:1:{s:7:"encoded";a:1:{i:0;a:5:{s:4:"data";s:18756:"

Everybody working on Unix or in the database world stumbles over Oracle Berkeley DB every now and then. DB is an Open Source embedded database used by applications like OpenLDAP or Postfix. Traditionally it followed mostly a key-value access pattern. Now what caught my attention was the fact that the recently released DB 5.0 provides an SQLite-like C API with the promise of providing better concurrency and performance than regular SQLite. Time to give it a shot.

So I grabbed the source distribution, checked the documentation and saw that I shall use the --enable-sql_compat configure option so DB creates a libsqlite3.so which can be used by PHP's build system. On my system compiling this worked like a charm and after linking PHP against it the SQLite3 extension as well as th SQLite PDO driver worked as before. I didn't do any benchmarks, yet, but I would be interested to read some results. Oh and yes, according to the documentation there are a few minor changes in behavior, this should only affect very few PHP users though.

So the whole process after extracting DB and PHP sources looked like this:

$ cd db-5.0.21/build_unix
$ ../dist/configure --prefix=/opt/db-5.0.21 --enable-sql_compat --enable-cxx
$ make
# make install
$ cd ../..
$ cd php-src
$ ./configure --with-sqlite3=/opt/db-5.0.21 --with-pdo-sqlite=/opt/db-5.0.21 --with-other-php-options
$ make
# make install

Now you don't have to use the SQLite interface for using DB but can also use PHP's dba extension. In current releases of PHP PHP expects at most DB version number 4 for the DBA driver, but a patch was committed to PHP's svn repository so upcoming PHP releases can be build using the latest DB version by adding --with-db4=/opt/db-5.0.21 to PHP's configure line.

Now I can hear you complain: "three ways to access a database isn't enough!" - which is true as the SQLite3 extension as well as the PDO SQLite driver are limited to SQLite capabilities and DBA, as an abstraction layer for Berkeley DB-like databases, only offers some basic operations, too. But there's hope:

While browsing through the DB source distribution I stumbled over a directory called php_db4 which obviously caught my further attention, so the DB developers are bundling a DB-specific PHP extension wrapping DB's native C++ API in an PHP OO API. To try it simply follow the steps from the PHP manual for compiling custom extensions:

$ cd db-5.0.21/php_db4
$ phpize
$ ./configure --with-db4=/opt/db-5.0.21
$ make
# make install

And then a quick check using reflection to see what functionality is provided by this extension:

$ php -dextension=db4.so --re db4
Extension [  extension #45 db4 version 0.9 ] {

  - Constants [210] {
    Constant [ integer DB_VERSION_MAJOR ] { 5 }
    Constant [ integer DB_VERSION_MINOR ] { 0 }
    Constant [ integer DB_VERSION_PATCH ] { 21 }
    Constant [ string DB_VERSION_STRING ] { Berkeley DB 5.0.21: (March 30, 2010) }
    Constant [ integer DB_MAX_PAGES ] { 4294967295 }
    Constant [ integer DB_MAX_RECORDS ] { 4294967295 }
    Constant [ integer DB_DBT_APPMALLOC ] { 1 }
    Constant [ integer DB_DBT_ISSET ] { 8 }
    Constant [ integer DB_DBT_MALLOC ] { 16 }
    Constant [ integer DB_DBT_PARTIAL ] { 64 }
    Constant [ integer DB_DBT_REALLOC ] { 128 }
    Constant [ integer DB_DBT_USERMEM ] { 1024 }
    Constant [ integer DB_DBT_DUPOK ] { 4 }
    Constant [ integer DB_CREATE ] { 1 }
    Constant [ integer DB_CXX_NO_EXCEPTIONS ] { 2 }
    Constant [ integer DB_FORCE ] { 1 }
    Constant [ integer DB_NOMMAP ] { 8 }
    Constant [ integer DB_RDONLY ] { 1024 }
    Constant [ integer DB_RECOVER ] { 2 }
    Constant [ integer DB_MULTIVERSION ] { 4 }
    Constant [ integer DB_TXN_SNAPSHOT ] { 2 }
    Constant [ integer DB_THREAD ] { 16 }
    Constant [ integer DB_TRUNCATE ] { 32768 }
    Constant [ integer DB_TXN_NOSYNC ] { 1 }
    Constant [ integer DB_TXN_NOT_DURABLE ] { 2 }
    Constant [ integer DB_USE_ENVIRON ] { 4 }
    Constant [ integer DB_USE_ENVIRON_ROOT ] { 8 }
    Constant [ integer DB_AUTO_COMMIT ] { 256 }
    Constant [ integer DB_DIRTY_READ ] { 512 }
    Constant [ integer DB_DEGREE_2 ] { 1024 }
    Constant [ integer DB_READ_COMMITTED ] { 1024 }
    Constant [ integer DB_READ_UNCOMMITTED ] { 512 }
    Constant [ integer DB_NO_AUTO_COMMIT ] { 8192 }
    Constant [ integer DB_RPCCLIENT ] { 1 }
    Constant [ integer DB_INIT_CDB ] { 64 }
    Constant [ integer DB_INIT_LOCK ] { 128 }
    Constant [ integer DB_INIT_LOG ] { 256 }
    Constant [ integer DB_INIT_MPOOL ] { 512 }
    Constant [ integer DB_INIT_REP ] { 1024 }
    Constant [ integer DB_INIT_TXN ] { 2048 }
    Constant [ integer DB_JOINENV ] { 0 }
    Constant [ integer DB_LOCKDOWN ] { 4096 }
    Constant [ integer DB_PRIVATE ] { 8192 }
    Constant [ integer DB_RECOVER_FATAL ] { 16384 }
    Constant [ integer DB_SYSTEM_MEM ] { 65536 }
    Constant [ integer DB_EXCL ] { 64 }
    Constant [ integer DB_FCNTL_LOCKING ] { 2048 }
    Constant [ integer DB_RDWRMASTER ] { 16384 }
    Constant [ integer DB_WRITEOPEN ] { 65536 }
    Constant [ integer DB_TXN_NOWAIT ] { 16 }
    Constant [ integer DB_TXN_SYNC ] { 4 }
    Constant [ integer DB_ENCRYPT_AES ] { 1 }
    Constant [ integer DB_CDB_ALLDB ] { 64 }
    Constant [ integer DB_DIRECT_DB ] { 128 }
    Constant [ integer DB_NOLOCKING ] { 1024 }
    Constant [ integer DB_NOPANIC ] { 2048 }
    Constant [ integer DB_OVERWRITE ] { 4096 }
    Constant [ integer DB_PANIC_ENVIRONMENT ] { 8192 }
    Constant [ integer DB_REGION_INIT ] { 16384 }
    Constant [ integer DB_TIME_NOTGRANTED ] { 32768 }
    Constant [ integer DB_TXN_WRITE_NOSYNC ] { 32 }
    Constant [ integer DB_YIELDCPU ] { 65536 }
    Constant [ integer DB_UPGRADE ] { 1 }
    Constant [ integer DB_VERIFY ] { 2 }
    Constant [ integer DB_DIRECT ] { 16 }
    Constant [ integer DB_EXTENT ] { 64 }
    Constant [ integer DB_ODDFILESIZE ] { 128 }
    Constant [ integer DB_CHKSUM ] { 8 }
    Constant [ integer DB_DUP ] { 16 }
    Constant [ integer DB_DUPSORT ] { 4 }
    Constant [ integer DB_ENCRYPT ] { 1 }
    Constant [ integer DB_RECNUM ] { 64 }
    Constant [ integer DB_RENUMBER ] { 128 }
    Constant [ integer DB_REVSPLITOFF ] { 256 }
    Constant [ integer DB_SNAPSHOT ] { 512 }
    Constant [ integer DB_STAT_CLEAR ] { 1 }
    Constant [ integer DB_JOIN_NOSORT ] { 1 }
    Constant [ integer DB_AGGRESSIVE ] { 1 }
    Constant [ integer DB_NOORDERCHK ] { 2 }
    Constant [ integer DB_ORDERCHKONLY ] { 4 }
    Constant [ integer DB_PR_PAGE ] { 16 }
    Constant [ integer DB_PR_RECOVERYTEST ] { 32 }
    Constant [ integer DB_PRINTABLE ] { 8 }
    Constant [ integer DB_SALVAGE ] { 64 }
    Constant [ integer DB_REP_NOBUFFER ] { 2 }
    Constant [ integer DB_REP_PERMANENT ] { 4 }
    Constant [ integer DB_LOCKVERSION ] { 1 }
    Constant [ integer DB_FILE_ID_LEN ] { 20 }
    Constant [ integer DB_LOCK_NORUN ] { 0 }
    Constant [ integer DB_LOCK_DEFAULT ] { 1 }
    Constant [ integer DB_LOCK_EXPIRE ] { 2 }
    Constant [ integer DB_LOCK_MAXLOCKS ] { 3 }
    Constant [ integer DB_LOCK_MINLOCKS ] { 5 }
    Constant [ integer DB_LOCK_MINWRITE ] { 6 }
    Constant [ integer DB_LOCK_OLDEST ] { 7 }
    Constant [ integer DB_LOCK_RANDOM ] { 8 }
    Constant [ integer DB_LOCK_YOUNGEST ] { 9 }
    Constant [ integer DB_LOCK_NOWAIT ] { 1 }
    Constant [ integer DB_LOCK_RECORD ] { 2 }
    Constant [ integer DB_LOCK_SET_TIMEOUT ] { 4 }
    Constant [ integer DB_LOCK_SWITCH ] { 8 }
    Constant [ integer DB_LOCK_UPGRADE ] { 16 }
    Constant [ integer DB_HANDLE_LOCK ] { 1 }
    Constant [ integer DB_RECORD_LOCK ] { 2 }
    Constant [ integer DB_PAGE_LOCK ] { 3 }
    Constant [ integer DB_LOGVERSION ] { 17 }
    Constant [ integer DB_LOGOLDVER ] { 8 }
    Constant [ integer DB_LOGMAGIC ] { 264584 }
    Constant [ integer DB_ARCH_ABS ] { 1 }
    Constant [ integer DB_ARCH_DATA ] { 2 }
    Constant [ integer DB_ARCH_LOG ] { 4 }
    Constant [ integer DB_ARCH_REMOVE ] { 8 }
    Constant [ integer DB_FLUSH ] { 1 }
    Constant [ integer DB_LOG_CHKPNT ] { 2 }
    Constant [ integer DB_LOG_COMMIT ] { 4 }
    Constant [ integer DB_LOG_NOCOPY ] { 8 }
    Constant [ integer DB_LOG_NOT_DURABLE ] { 16 }
    Constant [ integer DB_LOG_WRNOSYNC ] { 32 }
    Constant [ integer DB_user_BEGIN ] { 10000 }
    Constant [ integer DB_debug_FLAG ] { 2147483648 }
    Constant [ integer DB_LOG_DISK ] { 1 }
    Constant [ integer DB_LOG_LOCKED ] { 2 }
    Constant [ integer DB_LOG_SILENT_ERR ] { 4 }
    Constant [ integer DB_MPOOL_CREATE ] { 1 }
    Constant [ integer DB_MPOOL_LAST ] { 16 }
    Constant [ integer DB_MPOOL_NEW ] { 32 }
    Constant [ integer DB_MPOOL_DIRTY ] { 2 }
    Constant [ integer DB_MPOOL_DISCARD ] { 1 }
    Constant [ integer DB_MPOOL_NOFILE ] { 1 }
    Constant [ integer DB_MPOOL_UNLINK ] { 2 }
    Constant [ integer DB_TXNVERSION ] { 1 }
    Constant [ integer DB_GID_SIZE ] { 128 }
    Constant [ integer DB_EID_BROADCAST ] { -1 }
    Constant [ integer DB_EID_INVALID ] { -2 }
    Constant [ integer DB_REP_CLIENT ] { 1 }
    Constant [ integer DB_REP_MASTER ] { 2 }
    Constant [ integer DB_RENAMEMAGIC ] { 198656 }
    Constant [ integer DB_BTREEVERSION ] { 9 }
    Constant [ integer DB_BTREEOLDVER ] { 8 }
    Constant [ integer DB_BTREEMAGIC ] { 340322 }
    Constant [ integer DB_HASHVERSION ] { 9 }
    Constant [ integer DB_HASHOLDVER ] { 7 }
    Constant [ integer DB_HASHMAGIC ] { 398689 }
    Constant [ integer DB_QAMVERSION ] { 4 }
    Constant [ integer DB_QAMOLDVER ] { 3 }
    Constant [ integer DB_QAMMAGIC ] { 270931 }
    Constant [ integer DB_AFTER ] { 1 }
    Constant [ integer DB_APPEND ] { 2 }
    Constant [ integer DB_BEFORE ] { 3 }
    Constant [ integer DB_CONSUME ] { 4 }
    Constant [ integer DB_CONSUME_WAIT ] { 5 }
    Constant [ integer DB_CURRENT ] { 6 }
    Constant [ integer DB_FAST_STAT ] { 1 }
    Constant [ integer DB_FIRST ] { 7 }
    Constant [ integer DB_GET_BOTH ] { 8 }
    Constant [ integer DB_GET_BOTHC ] { 9 }
    Constant [ integer DB_GET_BOTH_RANGE ] { 10 }
    Constant [ integer DB_GET_RECNO ] { 11 }
    Constant [ integer DB_JOIN_ITEM ] { 12 }
    Constant [ integer DB_KEYFIRST ] { 13 }
    Constant [ integer DB_KEYLAST ] { 14 }
    Constant [ integer DB_LAST ] { 15 }
    Constant [ integer DB_NEXT ] { 16 }
    Constant [ integer DB_NEXT_DUP ] { 17 }
    Constant [ integer DB_NEXT_NODUP ] { 18 }
    Constant [ integer DB_NODUPDATA ] { 19 }
    Constant [ integer DB_NOOVERWRITE ] { 20 }
    Constant [ integer DB_NOSYNC ] { 21 }
    Constant [ integer DB_POSITION ] { 23 }
    Constant [ integer DB_PREV ] { 24 }
    Constant [ integer DB_PREV_NODUP ] { 26 }
    Constant [ integer DB_SET ] { 27 }
    Constant [ integer DB_SET_LOCK_TIMEOUT ] { 1 }
    Constant [ integer DB_SET_RANGE ] { 28 }
    Constant [ integer DB_SET_RECNO ] { 29 }
    Constant [ integer DB_SET_TXN_NOW ] { 8 }
    Constant [ integer DB_SET_TXN_TIMEOUT ] { 2 }
    Constant [ integer DB_UPDATE_SECONDARY ] { 30 }
    Constant [ integer DB_WRITECURSOR ] { 8 }
    Constant [ integer DB_WRITELOCK ] { 16 }
    Constant [ integer DB_OPFLAGS_MASK ] { 255 }
    Constant [ integer DB_MULTIPLE ] { 2048 }
    Constant [ integer DB_MULTIPLE_KEY ] { 16384 }
    Constant [ integer DB_RMW ] { 8192 }
    Constant [ integer DB_DONOTINDEX ] { -30998 }
    Constant [ integer DB_KEYEMPTY ] { -30996 }
    Constant [ integer DB_KEYEXIST ] { -30995 }
    Constant [ integer DB_LOCK_DEADLOCK ] { -30994 }
    Constant [ integer DB_LOCK_NOTGRANTED ] { -30993 }
    Constant [ integer DB_NOSERVER ] { -30990 }
    Constant [ integer DB_NOSERVER_HOME ] { -30989 }
    Constant [ integer DB_NOSERVER_ID ] { -30988 }
    Constant [ integer DB_NOTFOUND ] { -30987 }
    Constant [ integer DB_OLD_VERSION ] { -30986 }
    Constant [ integer DB_PAGE_NOTFOUND ] { -30985 }
    Constant [ integer DB_REP_DUPMASTER ] { -30984 }
    Constant [ integer DB_REP_HANDLE_DEAD ] { -30983 }
    Constant [ integer DB_REP_HOLDELECTION ] { -30982 }
    Constant [ integer DB_REP_ISPERM ] { -30980 }
    Constant [ integer DB_REP_NEWMASTER ] { -30893 }
    Constant [ integer DB_REP_NEWSITE ] { -30976 }
    Constant [ integer DB_REP_NOTPERM ] { -30975 }
    Constant [ integer DB_REP_UNAVAIL ] { -30974 }
    Constant [ integer DB_RUNRECOVERY ] { -30973 }
    Constant [ integer DB_SECONDARY_BAD ] { -30972 }
    Constant [ integer DB_VERIFY_BAD ] { -30970 }
    Constant [ integer DB_VERB_DEADLOCK ] { 1 }
    Constant [ integer DB_VERB_RECOVERY ] { 8 }
    Constant [ integer DB_VERB_REPLICATION ] { 32 }
    Constant [ integer DB_VERB_WAITSFOR ] { 32768 }
  }

  - Classes [4] {
    Class [  class db4txn ] {

      - Constants [0] {
      }

      - Static properties [0] {
      }

      - Static methods [0] {
      }

      - Properties [0] {
      }

      - Methods [8] {
        Method [  public method abort ] {
        }

        Method [  public method commit ] {
        }

        Method [  public method discard ] {
        }

        Method [  public method id ] {
        }

        Method [  public method set_timeout ] {
        }

        Method [  public method set_name ] {
        }

        Method [  public method get_name ] {
        }

        Method [  public method db4txn ] {
        }
      }
    }

    Class [  class db4cursor ] {

      - Constants [0] {
      }

      - Static properties [0] {
      }

      - Static methods [0] {
      }

      - Properties [0] {
      }

      - Methods [7] {
        Method [  public method close ] {
        }

        Method [  public method count ] {
        }

        Method [  public method del ] {
        }

        Method [  public method dup ] {
        }

        Method [  public method get ] {

          - Parameters [2] {
            Parameter #0 [  &$param0 ]
            Parameter #1 [  &$param1 ]
          }
        }

        Method [  public method put ] {
        }

        Method [  public method pget ] {

          - Parameters [2] {
            Parameter #0 [  &$param0 ]
            Parameter #1 [  &$param1 ]
          }
        }
      }
    }

    Class [  class db4 ] {

      - Constants [0] {
      }

      - Static properties [0] {
      }

      - Static methods [0] {
      }

      - Properties [0] {
      }

      - Methods [15] {
        Method [  public method db4 ] {
        }

        Method [  public method open ] {
        }

        Method [  public method close ] {
        }

        Method [  public method cursor ] {
        }

        Method [  public method del ] {
        }

        Method [  public method get ] {
        }

        Method [  public method get_encrypt_flags ] {
        }

        Method [  public method pget ] {
        }

        Method [  public method get_type ] {
        }

        Method [  public method join ] {
        }

        Method [  public method put ] {
        }

        Method [  public method set_encrypt ] {
        }

        Method [  public method stat ] {
        }

        Method [  public method sync ] {
        }

        Method [  public method truncate ] {
        }
      }
    }

    Class [  class db4env ] {

      - Constants [0] {
      }

      - Static properties [0] {
      }

      - Static methods [0] {
      }

      - Properties [0] {
      }

      - Methods [11] {
        Method [  public method db4env ] {
        }

        Method [  public method close ] {
        }

        Method [  public method dbremove ] {
        }

        Method [  public method dbrename ] {
        }

        Method [  public method get_encrypt ] {
        }

        Method [  public method open ] {
        }

        Method [  public method remove ] {
        }

        Method [  public method set_data_dir ] {
        }

        Method [  public method set_encrypt ] {
        }

        Method [  public method txn_begin ] {
        }

        Method [  public method txn_checkpoint ] {
        }
      }
    }
  }
}

The extension bundles some examples to get you started with Berkeley DB.

Even though I'd prefer a relational client-server database for most use cases I can image situations where a simple embedded solutions has its benefits, especially if combined with other Oracle Berkeley DB based applications.


PlanetMySQL Voting: Vote UP / Vote DOWN";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}s:32:"http://purl.org/dc/elements/1.1/";a:1:{s:7:"creator";a:1:{i:0;a:5:{s:4:"data";s:22:"Johannes Schlüter";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}}}i:32;a:6:{s:4:"data";s:33:" ";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";s:5:"child";a:2:{s:0:"";a:5:{s:5:"title";a:1:{i:0;a:5:{s:4:"data";s:69:"Sales en: ODBA Interview with FromDual about the MySQL/MariaDB future";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:4:"guid";a:1:{i:0;a:5:{s:4:"data";s:52:"http://www.fromdual.com/interview-about-mysql-future";s:7:"attribs";a:1:{s:0:"";a:1:{s:11:"isPermaLink";s:5:"false";}}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:4:"link";a:1:{i:0;a:5:{s:4:"data";s:52:"http://www.fromdual.com/interview-about-mysql-future";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:11:"description";a:1:{i:0;a:5:{s:4:"data";s:438:"Henrik Ingo from the Open Database Alliance (ODBA) did an interview with Oli Sennhauser of FromDual about the European MySQL database landscape and its future. If you want to know more about our opinion in this matter please read here: http://opendba.wordpress.com/2010/05/31/interview-oli-sennhauser-of-fromdual-about-european-database-landscape/. For more technical insight see also our former presentation: MySQL, where are you going?.";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:7:"pubDate";a:1:{i:0;a:5:{s:4:"data";s:31:"Thu, 03 Jun 2010 13:29:16 +0000";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}s:40:"http://purl.org/rss/1.0/modules/content/";a:1:{s:7:"encoded";a:1:{i:0;a:5:{s:4:"data";s:1137:"

Henrik Ingo from the Open Database Alliance (ODBA) did an interview with Oli Sennhauser of FromDual about the European MySQL database landscape and its future. If you want to know more about our opinion in this matter please read here: http://opendba.wordpress.com/2010/05/31/interview-oli-sennhauser-of-fromdual-about-european-database-landscape/.

For more technical insight see also our former presentation: MySQL, where are you going?.


PlanetMySQL Voting: Vote UP / Vote DOWN";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}}}i:33;a:6:{s:4:"data";s:53:" ";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";s:5:"child";a:3:{s:0:"";a:6:{s:5:"title";a:1:{i:0;a:5:{s:4:"data";s:41:"Southeast Linux Fest is around the corner";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:4:"guid";a:1:{i:0;a:5:{s:4:"data";s:33:"http://www.xaprb.com/blog/?p=1890";s:7:"attribs";a:1:{s:0:"";a:1:{s:11:"isPermaLink";s:5:"false";}}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:4:"link";a:1:{i:0;a:5:{s:4:"data";s:79:"http://www.xaprb.com/blog/2010/06/03/southeast-linux-fest-is-around-the-corner/";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:11:"description";a:1:{i:0;a:5:{s:4:"data";s:741:"If you’re near South Carolina next weekend, consider attending Southeast Linux Fest! There’s a list of illustrious speakers including several well-known in the database world: Joshua Drake and Andrew Dunstan (PostgreSQL), D. Richard Hipp (SQLite), and yes, yours truly (MySQL), plus a MySQL name that’s new to me: Brandon Checketts. There are a ton of non-database sessions too! Check out the full speaker & session list. This was a great show last year; I highly encourage everyone to attend. Related posts:Recap of Southeast Linux Fest 2009How Linux iostat computes its resultsHow to find per-process I/O statistics on LinuxHow to monitor server load on GNU/LinuxHow to auto-mount removable devices in GNU/Linux";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:7:"pubDate";a:1:{i:0;a:5:{s:4:"data";s:31:"Thu, 03 Jun 2010 12:37:34 +0000";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:8:"category";a:3:{i:0;a:5:{s:4:"data";s:11:"Conferences";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:1;a:5:{s:4:"data";s:10:"PostgreSQL";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:2;a:5:{s:4:"data";s:3:"SQL";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}s:40:"http://purl.org/rss/1.0/modules/content/";a:1:{s:7:"encoded";a:1:{i:0;a:5:{s:4:"data";s:1995:"

If you’re near South Carolina next weekend, consider attending Southeast Linux Fest! There’s a list of illustrious speakers including several well-known in the database world: Joshua Drake and Andrew Dunstan (PostgreSQL), D. Richard Hipp (SQLite), and yes, yours truly (MySQL), plus a MySQL name that’s new to me: Brandon Checketts. There are a ton of non-database sessions too! Check out the full speaker & session list. This was a great show last year; I highly encourage everyone to attend.

Related posts:

  1. Recap of Southeast Linux Fest 2009
  2. How Linux iostat computes its results
  3. How to find per-process I/O statistics on Linux
  4. How to monitor server load on GNU/Linux
  5. How to auto-mount removable devices in GNU/Linux


PlanetMySQL Voting: Vote UP / Vote DOWN";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}s:32:"http://purl.org/dc/elements/1.1/";a:1:{s:7:"creator";a:1:{i:0;a:5:{s:4:"data";s:22:"Baron Schwartz (xaprb)";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}}}i:34;a:6:{s:4:"data";s:73:" ";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";s:5:"child";a:3:{s:0:"";a:6:{s:5:"title";a:1:{i:0;a:5:{s:4:"data";s:36:"New CREATE TABLE performance record!";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:4:"guid";a:1:{i:0;a:5:{s:4:"data";s:40:"http://www.flamingspork.com/blog/?p=2036";s:7:"attribs";a:1:{s:0:"";a:1:{s:11:"isPermaLink";s:5:"false";}}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:4:"link";a:1:{i:0;a:5:{s:4:"data";s:80:"http://www.flamingspork.com/blog/2010/06/03/new-create-table-performance-record/";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:11:"description";a:1:{i:0;a:5:{s:4:"data";s:1886:"4 min 20 sec So next time somebody complains about NDB taking a long time in CREATE TABLE, you’re welcome to point them to this :) A single CREATE TABLE statement It had ONE column It was an ENUM column. With 70,000 possible values. It was 605kb of SQL. It ran on Drizzle This was to test if you could create an ENUM column with greater than 216 possible values (you’re not supposed to be able to) – bug 589031 has been filed. How does it compare to MySQL? Well… there are other problems (Bug 54194 – ENUM limit of 65535 elements isn’t true filed). Since we don’t have any limitations in Drizzle due to the FRM file format, we actually get to execute the CREATE TABLE statement. Still, why did this take four and a half minutes? I luckily managed to run poor man’s profiler during query execution. I very easily found out that I had this thread constantly running check_duplicates_in_interval(), which does a stupid linear search for duplicates. It turns out, that for 70,000 items, this takes approximately four minutes and 19.5 seconds. Bug 589055 CREATE TABLE with ENUM fields with large elements takes forever (where forever is defined as a bit over four minutes) filed. So I replaced check_duplicates_in_interval() with a implementation using a hash table (boost::unordered_set actually) as I wasn’t quite immediately in the mood for ripping out all of TYPELIB from the server. I can now run the CREATE TABLE statement in less than half a second. So now, I can run my test case in much less time and indeed check for correct behaviour rather quickly. I do have an urge to find out how big I can get a valid table definition file to though…. should be over 32MB… Share this on Facebook Tweet This! Share this on del.icio.us Digg this! Post on Google Buzz ";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:7:"pubDate";a:1:{i:0;a:5:{s:4:"data";s:31:"Thu, 03 Jun 2010 07:40:10 +0000";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:8:"category";a:7:{i:0;a:5:{s:4:"data";s:7:"drizzle";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:1;a:5:{s:4:"data";s:5:"mysql";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:2;a:5:{s:4:"data";s:5:"boost";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:3;a:5:{s:4:"data";s:3:"bug";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:4;a:5:{s:4:"data";s:12:"CREATE TABLE";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:5;a:5:{s:4:"data";s:4:"enum";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:6;a:5:{s:4:"data";s:3:"sql";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}s:40:"http://purl.org/rss/1.0/modules/content/";a:1:{s:7:"encoded";a:1:{i:0;a:5:{s:4:"data";s:3685:"

4 min 20 sec

So next time somebody complains about NDB taking a long time in CREATE TABLE, you’re welcome to point them to this :)

This was to test if you could create an ENUM column with greater than 216 possible values (you’re not supposed to be able to) – bug 589031 has been filed.

How does it compare to MySQL? Well… there are other problems (Bug 54194 – ENUM limit of 65535 elements isn’t true filed). Since we don’t have any limitations in Drizzle due to the FRM file format, we actually get to execute the CREATE TABLE statement.

Still, why did this take four and a half minutes? I luckily managed to run poor man’s profiler during query execution. I very easily found out that I had this thread constantly running check_duplicates_in_interval(), which does a stupid linear search for duplicates. It turns out, that for 70,000 items, this takes approximately four minutes and 19.5 seconds. Bug 589055 CREATE TABLE with ENUM fields with large elements takes forever (where forever is defined as a bit over four minutes) filed.

So I replaced check_duplicates_in_interval() with a implementation using a hash table (boost::unordered_set actually) as I wasn’t quite immediately in the mood for ripping out all of TYPELIB from the server. I can now run the CREATE TABLE statement in less than half a second.

So now, I can run my test case in much less time and indeed check for correct behaviour rather quickly.

I do have an urge to find out how big I can get a valid table definition file to though…. should be over 32MB…


PlanetMySQL Voting: Vote UP / Vote DOWN";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}s:32:"http://purl.org/dc/elements/1.1/";a:1:{s:7:"creator";a:1:{i:0;a:5:{s:4:"data";s:13:"Stewart Smith";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}}}i:35;a:6:{s:4:"data";s:53:" ";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";s:5:"child";a:3:{s:0:"";a:6:{s:5:"title";a:1:{i:0;a:5:{s:4:"data";s:33:"Choosing MySQL boolean data types";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:4:"guid";a:1:{i:0;a:5:{s:4:"data";s:36:"http://code.openark.org/blog/?p=2181";s:7:"attribs";a:1:{s:0:"";a:1:{s:11:"isPermaLink";s:5:"false";}}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:4:"link";a:1:{i:0;a:5:{s:4:"data";s:68:"http://code.openark.org/blog/mysql/choosing-mysql-boolean-data-types";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:11:"description";a:1:{i:0;a:5:{s:4:"data";s:3253:"How do you implement True/False columns? There are many ways to do it, each with its own pros and cons. ENUM Create you column as ENUM(‘F’, ‘T’), or ENUM(‘N’,'Y’) or ENUM(‘0′, ‘1′). This is the method used in the mysql tables (e.g. mysql.user privileges table). It’s very simple and intuitive. It truly restricts the values to just two options, which serves well. It’s compact (just one byte). A couple disadvantages to this method: Enums are represented by numerical values (which is good) and start with 1 instead of 0. This means ‘F’ is 1, and ‘T’ is 2, and they both translate to True when directly used in a booleanic expression (e.g. IF(val, ‘True’, ‘False’) always yields ‘True’) There’s no real convention. Is it ‘Y’/'N’? ‘T’/'F’? ‘P’/'N’? ‘1′/’0′? CHAR(1) Simple again. Proposed values are, as before, ‘F’, ‘T’ etc. This time there’s no way to limit the range of values. You cannot (in MySQL, unless using triggers) prevent an ‘X’. Watch out for the charset! If it’s utf8 you pay with 3 bytes instead of just 1. And, again, ‘T’, ‘F’, ‘Y’, ‘N’ values all evaluate as True. It is possible to use the zero-valued character, but it defeats the purpose of using CHAR. CHAR(0) Many are unaware that it’s even valid to make this definition. What does it mean? Take a look at the following table: CREATE TABLE `t1` ( `bval` char(0) DEFAULT NULL ); mysql> INSERT INTO t1 VALUES (''); mysql> INSERT INTO t1 VALUES (''); mysql> INSERT INTO t1 VALUES (NULL); mysql> SELECT * FROM t1; +------+ | bval | +------+ |      | |      | | NULL | +------+ NULLable columns cause for an additional storage per row. There’s one bit per NULLable column which notes down whether the column’s value is NULL or not. If you only have one NULLable column, you must pay for this bit with 1 byte. If you have two NULLable columns, you still only pay with 1 byte. Furthermore: mysql> SELECT bval IS NOT NULL FROM t1; +------------------+ | bval IS NOT NULL | +------------------+ |                1 | |                1 | |                0 | +------------------+ So this plays somewhat nicely into booleanic expressions. However, this method is unintuitive and confusing. I personally don’t use it. TINYINT With integer values, we can get down to 0 and 1. With TINYINT, we only pay with 1 byte of storage. As with CHAR(1), we cannot prevent anyone from INSERTing other values. But that doesn’t really matter, if we’re willing to accept that 0 evaluates as False, and all other values as True. In this case, boolean expressions work very well with your column values. BOOL/BOOLEAN These are just synonyms to TINYINT. I like to define my boolean values as such. Alas, when issuing a SHOW CREATE TABLE the definition is just a normal TINYINT. Still, it is clearer to look at if you’re storing your table schema under your version control.";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:7:"pubDate";a:1:{i:0;a:5:{s:4:"data";s:31:"Thu, 03 Jun 2010 05:24:11 +0000";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:8:"category";a:3:{i:0;a:5:{s:4:"data";s:5:"MySQL";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:1;a:5:{s:4:"data";s:10:"Data Types";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:2;a:5:{s:4:"data";s:3:"SQL";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}s:40:"http://purl.org/rss/1.0/modules/content/";a:1:{s:7:"encoded";a:1:{i:0;a:5:{s:4:"data";s:4376:"

How do you implement True/False columns?

There are many ways to do it, each with its own pros and cons.

ENUM

Create you column as ENUM(‘F’, ‘T’), or ENUM(‘N’,'Y’) or ENUM(‘0′, ‘1′).

This is the method used in the mysql tables (e.g. mysql.user privileges table). It’s very simple and intuitive. It truly restricts the values to just two options, which serves well. It’s compact (just one byte).

A couple disadvantages to this method:

  1. Enums are represented by numerical values (which is good) and start with 1 instead of 0. This means ‘F’ is 1, and ‘T’ is 2, and they both translate to True when directly used in a booleanic expression (e.g. IF(val, ‘True’, ‘False’) always yields ‘True’)
  2. There’s no real convention. Is it ‘Y’/'N’? ‘T’/'F’? ‘P’/'N’? ‘1′/’0′?

CHAR(1)

Simple again. Proposed values are, as before, ‘F’, ‘T’ etc. This time there’s no way to limit the range of values. You cannot (in MySQL, unless using triggers) prevent an ‘X’.

Watch out for the charset! If it’s utf8 you pay with 3 bytes instead of just 1. And, again, ‘T’, ‘F’, ‘Y’, ‘N’ values all evaluate as True. It is possible to use the zero-valued character, but it defeats the purpose of using CHAR.

CHAR(0)

Many are unaware that it’s even valid to make this definition. What does it mean? Take a look at the following table:

CREATE TABLE `t1` (
 `bval` char(0) DEFAULT NULL
);
mysql> INSERT INTO t1 VALUES ('');
mysql> INSERT INTO t1 VALUES ('');
mysql> INSERT INTO t1 VALUES (NULL);

mysql> SELECT * FROM t1;
+------+
| bval |
+------+
|      |
|      |
| NULL |
+------+

NULLable columns cause for an additional storage per row. There’s one bit per NULLable column which notes down whether the column’s value is NULL or not. If you only have one NULLable column, you must pay for this bit with 1 byte. If you have two NULLable columns, you still only pay with 1 byte.

Furthermore:

mysql> SELECT bval IS NOT NULL FROM t1;
+------------------+
| bval IS NOT NULL |
+------------------+
|                1 |
|                1 |
|                0 |
+------------------+

So this plays somewhat nicely into booleanic expressions.

However, this method is unintuitive and confusing. I personally don’t use it.

TINYINT

With integer values, we can get down to 0 and 1. With TINYINT, we only pay with 1 byte of storage. As with CHAR(1), we cannot prevent anyone from INSERTing other values. But that doesn’t really matter, if we’re willing to accept that 0 evaluates as False, and all other values as True. In this case, boolean expressions work very well with your column values.

BOOL/BOOLEAN

These are just synonyms to TINYINT. I like to define my boolean values as such. Alas, when issuing a SHOW CREATE TABLE the definition is just a normal TINYINT. Still, it is clearer to look at if you’re storing your table schema under your version control.


PlanetMySQL Voting: Vote UP / Vote DOWN";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}s:32:"http://purl.org/dc/elements/1.1/";a:1:{s:7:"creator";a:1:{i:0;a:5:{s:4:"data";s:12:"Shlomi Noach";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}}}i:36;a:6:{s:4:"data";s:58:" ";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";s:5:"child";a:3:{s:0:"";a:6:{s:5:"title";a:1:{i:0;a:5:{s:4:"data";s:53:"FlashCache: tpcc workload with FusionIO card as cache";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:4:"guid";a:1:{i:0;a:5:{s:4:"data";s:43:"http://www.mysqlperformanceblog.com/?p=2941";s:7:"attribs";a:1:{s:0:"";a:1:{s:11:"isPermaLink";s:5:"false";}}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:4:"link";a:1:{i:0;a:5:{s:4:"data";s:100:"http://www.mysqlperformanceblog.com/2010/06/02/flashcache-tpcc-workload-with-fusionio-card-as-cache/";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:11:"description";a:1:{i:0;a:5:{s:4:"data";s:1418:"This run is very similar what I had on Intel SSD X25-M card, but now I use FusionIO 80GB SLC card. I chose this card as smallest available card (and therefore cheapest. On Dell.com you can see it for about $3K). There is also FusionIO IO-Xtreme 80GB card, which is however MLC based and it could be not best choice for FlashCache usage ( as there high write rate on FlashCache for both reading and writing to/from disks, so lifetime could be short). Also Facebook team released WriteThrough module for FlashCache, which could be good trade-off if you want extra warranty for data consistency and your load is mostly read-bound, so I tested this mode also. All setup is similar to previous post, so let me just post the results with FlashCache on FusionIO in 20% dirty page, 80% dirty pages and write-through modes. I used full 80GB for caching ( total size of data is about 100GB). Conclusions from the graph: with 80% dirty page we have about 4x better throughput ( comparing to RAID). Write-through mode is about 2x gain, but remember that load is very write intensive and all benefits in write-through mode come only from cached reads, so it is pretty good for this scenario On this post I finish my runs on FlashCache for now and I think it may be considered for real usage, at least you may evaluate how it works on your workloads. Entry posted by Vadim | 11 comments Add to: | | | | ";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:7:"pubDate";a:1:{i:0;a:5:{s:4:"data";s:31:"Thu, 03 Jun 2010 02:03:03 +0000";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:8:"category";a:4:{i:0;a:5:{s:4:"data";s:3:"SSD";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:1;a:5:{s:4:"data";s:10:"benchmarks";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:2;a:5:{s:4:"data";s:8:"fusionio";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:3;a:5:{s:4:"data";s:5:"mysql";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}s:40:"http://purl.org/rss/1.0/modules/content/";a:1:{s:7:"encoded";a:1:{i:0;a:5:{s:4:"data";s:4428:"

This run is very similar what I had on Intel SSD X25-M card, but now I use FusionIO 80GB SLC card. I chose this card as smallest available card (and therefore cheapest. On Dell.com you can see it for about $3K). There is also FusionIO IO-Xtreme 80GB card, which is however MLC based and it could be not best choice for FlashCache usage ( as there high write rate on FlashCache for both reading and writing to/from disks, so lifetime could be short).

Also Facebook team released WriteThrough module for FlashCache, which could be good trade-off if you want extra warranty for data consistency and your load is mostly read-bound, so I tested this mode also.

All setup is similar to previous post, so let me just post the results with FlashCache on FusionIO in 20% dirty page, 80% dirty pages and write-through modes. I used full 80GB for caching ( total size of data is about 100GB).

Conclusions from the graph:

On this post I finish my runs on FlashCache for now and I think it may be considered for real usage, at least you may evaluate how it works on your workloads.

Entry posted by Vadim | 11 comments

Add to: delicious | digg | reddit | netscape | Google Bookmarks


PlanetMySQL Voting: Vote UP / Vote DOWN";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}s:32:"http://purl.org/dc/elements/1.1/";a:1:{s:7:"creator";a:1:{i:0;a:5:{s:4:"data";s:22:"MySQL Performance Blog";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}}}i:37;a:6:{s:4:"data";s:58:" ";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";s:5:"child";a:3:{s:0:"";a:6:{s:5:"title";a:1:{i:0;a:5:{s:4:"data";s:27:"PBXT in tpcc-like benchmark";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:4:"guid";a:1:{i:0;a:5:{s:4:"data";s:43:"http://www.mysqlperformanceblog.com/?p=2930";s:7:"attribs";a:1:{s:0:"";a:1:{s:11:"isPermaLink";s:5:"false";}}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:4:"link";a:1:{i:0;a:5:{s:4:"data";s:75:"http://www.mysqlperformanceblog.com/2010/06/02/pbxt-in-tpcc-like-benchmark/";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:11:"description";a:1:{i:0;a:5:{s:4:"data";s:1577:"Finally I was able to run PBXT 1.0.11 pre-GA in tpcc-like workload, apparently there was bug with did not allow me to get the result earlier, and I am happy to see that PBXT team managed it. For initial runs I took tpcc 100 warehouses ( about 10GB of data) which fully fits into memory (32 GB on server), and compared 1 and 16 users in MySQL-5.1.46/PBXT and Percona Server / XtraDB – 5.1.45-rel10.2. As workload is totally memory based it will show how PBXT scales in CPU-bond cases on 16 cores systems. As storage system it was Intel SSD X25-M card. While full results and config are on Wiki: http://www.percona.com/docs/wiki/benchmark:pbxt:tpcc:start there are graphs for 1 user: and 16 users: Interesting to see that in case with 1 user the maximal throughput in PBXT is about 1.5x better XtraDB, but there periodical drops which are very similar to periodical drops in InnoDB without adaptive checkpointing, and I guess it is also related to checkpoint activity. The final results are also better for PBXT: 5785.567 TpmC vs 4905.967 TpmC ( XtraDB) For 16 threads final result is: 26129.350 TpmC for PBXT and 29485.518 TpmC for XtraDB , and from the graph you can see that the maximal throughput is about identical, while PBXT spends more time in drops area. Again it looks like PBXT are not fully keeping up with checkpoint activity and I am looking PBXT addresses this problem also. Beside this issue PBXT looks pretty good and in next round I am going to run IO intensive workloads. Entry posted by Vadim | 10 comments Add to: | | | | ";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:7:"pubDate";a:1:{i:0;a:5:{s:4:"data";s:31:"Thu, 03 Jun 2010 00:19:47 +0000";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:8:"category";a:4:{i:0;a:5:{s:4:"data";s:3:"SSD";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:1;a:5:{s:4:"data";s:10:"benchmarks";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:2;a:5:{s:4:"data";s:5:"mysql";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:3;a:5:{s:4:"data";s:4:"pbxt";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}s:40:"http://purl.org/rss/1.0/modules/content/";a:1:{s:7:"encoded";a:1:{i:0;a:5:{s:4:"data";s:4352:"

Finally I was able to run PBXT 1.0.11 pre-GA in tpcc-like workload, apparently there was bug with did not allow me to get the result earlier, and I am happy to see that PBXT team managed it.

For initial runs I took tpcc 100 warehouses ( about 10GB of data) which fully fits into memory (32 GB on server),
and compared 1 and 16 users in MySQL-5.1.46/PBXT and Percona Server / XtraDB – 5.1.45-rel10.2. As workload is totally memory based it will show how PBXT scales in CPU-bond cases on 16 cores systems.

As storage system it was Intel SSD X25-M card.

While full results and config are on Wiki:
http://www.percona.com/docs/wiki/benchmark:pbxt:tpcc:start

there are graphs for 1 user:

and 16 users:

Interesting to see that in case with 1 user the maximal throughput in PBXT is about 1.5x better XtraDB, but
there periodical drops which are very similar to periodical drops in InnoDB without adaptive checkpointing, and I guess it is also related to checkpoint activity.
The final results are also better for PBXT: 5785.567 TpmC vs 4905.967 TpmC ( XtraDB)

For 16 threads final result is: 26129.350 TpmC for PBXT and 29485.518 TpmC for XtraDB , and from the graph you can see that the maximal throughput is about identical, while PBXT spends more time in drops area. Again it looks like PBXT are not fully keeping up with checkpoint activity and I am looking PBXT addresses this problem also. Beside this issue PBXT looks pretty good and in next round I am going to run IO intensive workloads.

Entry posted by Vadim | 10 comments

Add to: delicious | digg | reddit | netscape | Google Bookmarks


PlanetMySQL Voting: Vote UP / Vote DOWN";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}s:32:"http://purl.org/dc/elements/1.1/";a:1:{s:7:"creator";a:1:{i:0;a:5:{s:4:"data";s:22:"MySQL Performance Blog";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}}}i:38;a:6:{s:4:"data";s:78:" ";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";s:5:"child";a:3:{s:0:"";a:6:{s:5:"title";a:1:{i:0;a:5:{s:4:"data";s:22:"User Defined Variables";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:4:"guid";a:1:{i:0;a:5:{s:4:"data";s:30:"http://www.mysqldiary.com/?p=7";s:7:"attribs";a:1:{s:0:"";a:1:{s:11:"isPermaLink";s:5:"false";}}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:4:"link";a:1:{i:0;a:5:{s:4:"data";s:49:"http://www.mysqldiary.com/user-defined-variables/";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:11:"description";a:1:{i:0;a:5:{s:4:"data";s:334:"When I have discovered MySQL User-defined variables at the first time, I didn’t pay much attention to them and didn’t realize their strength. At first, one can think that they are just one of the many unnoticed MySQL features. Well, I was wrong. Here, I will try to summarize their strength, though I think there is [...]";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:7:"pubDate";a:1:{i:0;a:5:{s:4:"data";s:31:"Wed, 02 Jun 2010 21:04:19 +0000";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:8:"category";a:8:{i:0;a:5:{s:4:"data";s:13:"MySQL feature";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:1;a:5:{s:4:"data";s:16:"case sensitivity";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:2;a:5:{s:4:"data";s:14:"mysql features";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:3;a:5:{s:4:"data";s:11:"mysql query";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:4;a:5:{s:4:"data";s:16:"procedural logic";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:5;a:5:{s:4:"data";s:16:"relational logic";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:6;a:5:{s:4:"data";s:21:"User defined variable";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:7;a:5:{s:4:"data";s:9:"variables";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}s:40:"http://purl.org/rss/1.0/modules/content/";a:1:{s:7:"encoded";a:1:{i:0;a:5:{s:4:"data";s:547:"When I have discovered MySQL User-defined variables at the first time, I didn’t pay much attention to them and didn’t realize their strength. At first, one can think that they are just one of the many unnoticed MySQL features. Well, I was wrong. Here, I will try to summarize their strength, though I think there is [...]
PlanetMySQL Voting: Vote UP / Vote DOWN";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}s:32:"http://purl.org/dc/elements/1.1/";a:1:{s:7:"creator";a:1:{i:0;a:5:{s:4:"data";s:10:"Ilan Hazan";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}}}i:39;a:6:{s:4:"data";s:58:" ";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";s:5:"child";a:3:{s:0:"";a:6:{s:5:"title";a:1:{i:0;a:5:{s:4:"data";s:37:"MySQL Workbench 5.2.22 RC 2 Available";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:4:"guid";a:1:{i:0;a:5:{s:4:"data";s:26:"http://wb.mysql.com/?p=581";s:7:"attribs";a:1:{s:0:"";a:1:{s:11:"isPermaLink";s:5:"false";}}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:4:"link";a:1:{i:0;a:5:{s:4:"data";s:26:"http://wb.mysql.com/?p=581";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:11:"description";a:1:{i:0;a:5:{s:4:"data";s:2800:"Dear MySQL Users, We’re happy to announce the second Release Candidate (RC) of MySQL Workbench. Version 5.2.22 includes fixes for 76 bugs. In case you didn’t know – MySQL Workbench is a modular product and is fully pluggable and scriptable. If you want to write a plug-in for Workbench 5.2, then stay tuned as we will be updating the documentation in the next week with all the details and examples you need on how to write your own plug-ins. Plug-ins are great for extending or adding to workbench or for automating repetitive tasks. You can do things like adding to menus, exporting your Workbench models custom tailored for your coding environment, adding your own forms, and more. Python is our preferred scripting language but for simple scripts the Lua language can be used as well. The first plug-in we added is called MySQL Doc Library. It can be used to read the MySQL Documentation offline and features many convenient features such as full text search. Currently, only the MySQL Workbench manual is included, but material and features will be added over time. MySQL Workbench 5.2 RC provides: Data Modeling Query (upgrade from MySQL Query Browser) Admin (upgrade from MySQL Administrator) if you are a current user of MySQL Query Browser or MySQL Administrator, we look forward to your feedback on all the new capabilities we are delivering in a single unified MySQL Workbench As always, you will find binaries for the various platforms on our download pages. Please get your copy from our Download http://dev.mysql.com/downloads/workbench/ To get started quickly, please take a look at this short tutorial. MySQL Workbench 5.2 RC Tutorial http://wb.mysql.com/?p=406 Please be aware that this release is still a in-development version – so please don’t use it on your production servers! Also note, MySQL Workbench files saved with version 5.2 cannot be opened with previous versions of our program. The files for several platforms have been pushed to our main server and should be available on our mirrors. Blog postings and general information – including build instructions for Linux – can be found on our Workbench Developer Central site. Workbench Developer Central http://wb.mysql.com Workbench Documentation and details on changes between releases can be found on these pages http://dev.mysql.com/doc/workbench/en/index.html http://dev.mysql.com/doc/workbench/en/wb-change-history.html If you need any additional info or help please get in touch with us. Post in our forums, leave comments on our blog pages or if you want to talk to us directly you can visit us on our IRC channel #workbench on irc.freenode.net. Again, thank you for trying out the Workbench development versions, we look forward to your feedback and bug reports. - The MySQL Workbench Team";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:7:"pubDate";a:1:{i:0;a:5:{s:4:"data";s:31:"Wed, 02 Jun 2010 20:49:00 +0000";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:8:"category";a:4:{i:0;a:5:{s:4:"data";s:13:"Announcements";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:1;a:5:{s:4:"data";s:5:"Linux";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:2;a:5:{s:4:"data";s:3:"Mac";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:3;a:5:{s:4:"data";s:7:"Windows";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}s:40:"http://purl.org/rss/1.0/modules/content/";a:1:{s:7:"encoded";a:1:{i:0;a:5:{s:4:"data";s:3545:"

Dear MySQL Users,

We’re happy to announce the second Release Candidate (RC) of MySQL Workbench.
Version 5.2.22 includes fixes for 76 bugs.

In case you didn’t know – MySQL Workbench is a modular product and is fully pluggable and scriptable. If you want to write a plug-in for Workbench 5.2, then stay tuned as we will be updating the documentation in the next week with all the details and examples you need on how to write your own plug-ins.

Plug-ins are great for extending or adding to workbench or for automating repetitive tasks. You can do things like adding to menus, exporting your Workbench models custom tailored for your coding environment, adding your own forms, and more. Python is our preferred scripting language but for simple scripts the Lua language can be used as well.

The first plug-in we added is called MySQL Doc Library. It can be used to read the MySQL Documentation offline and features many convenient features such as full text search. Currently, only the MySQL Workbench manual is included, but material and features will be added over time.

MySQL Workbench 5.2 RC provides:

  1. Data Modeling
  2. Query (upgrade from MySQL Query Browser)
  3. Admin (upgrade from MySQL Administrator)

if you are a current user of MySQL Query Browser or MySQL Administrator, we look forward to your feedback on all the new capabilities we are delivering in a single unified MySQL Workbench

As always, you will find binaries for the various platforms on our download pages.

Please get your copy from our Download

http://dev.mysql.com/downloads/workbench/

To get started quickly, please take a look at this short tutorial.

MySQL Workbench 5.2 RC Tutorial

http://wb.mysql.com/?p=406

Please be aware that this release is still a in-development version – so please don’t use it on your production servers! Also note, MySQL Workbench files saved with version 5.2 cannot be opened with previous versions of our program.

The files for several platforms have been pushed to our main server and should be available on our mirrors.

Blog postings and general information – including build instructions for Linux – can be found on our Workbench Developer Central site.

Workbench Developer Central

http://wb.mysql.com

Workbench Documentation and details on changes between releases can be found on these pages

http://dev.mysql.com/doc/workbench/en/index.html
http://dev.mysql.com/doc/workbench/en/wb-change-history.html

If you need any additional info or help please get in touch with us.
Post in our forums, leave comments on our blog pages or if you want to talk to us directly you can visit us on our IRC channel #workbench on irc.freenode.net.

Again, thank you for trying out the Workbench development versions, we look forward to your feedback and bug reports.

- The MySQL Workbench Team


PlanetMySQL Voting: Vote UP / Vote DOWN";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}s:32:"http://purl.org/dc/elements/1.1/";a:1:{s:7:"creator";a:1:{i:0;a:5:{s:4:"data";s:18:"The Workbench Team";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}}}i:40;a:6:{s:4:"data";s:48:" ";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";s:5:"child";a:3:{s:0:"";a:6:{s:5:"title";a:1:{i:0;a:5:{s:4:"data";s:46:"mysqlnd plugins: alternative to MySQL Proxy ?!";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:4:"guid";a:1:{i:0;a:5:{s:4:"data";s:32:"http://blog.ulf-wendel.de/?p=284";s:7:"attribs";a:1:{s:0:"";a:1:{s:11:"isPermaLink";s:5:"false";}}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:4:"link";a:1:{i:0;a:5:{s:4:"data";s:32:"http://blog.ulf-wendel.de/?p=284";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:11:"description";a:1:{i:0;a:5:{s:4:"data";s:8342:" The mysqlnd plugin API is a well hidden gem of mysqlnd. Mysqlnd plugins operate on a layer between PHP applications and the MySQL server. This is comparable to MySQL Proxy. MySQL Proxy operates on a layer between any MySQL client application, for example, a PHP application and, the MySQL server. Plugins can take over classical MySQL Proxy tasks such as Load Balancing, Monitoring and Performance optimizations. But due to the different architecture and location mysqlnd plugins do not share some common MySQL Proxy annoyances: no single point of failure, no dedicated proxy server to deploy, no new programming language to learn (Lua). Slides from the IPC Spring conference This blog posting and today’s presentation at the IPC Spring 2010, a PHP conference in Berlin, are the first public information on the new toy. Enjoy the slides or continue reading the blog posting. The PHP mysqlnd plugin talk - plugins an alternative to MySQL Proxy View more presentations from Ulf Wendel. The mysqlnd plugin talk (PDF) The mysqlnd plugin talk (OpenOffice.org presentation - *.odp) What mysqlnd plugins can do! A mysqlnd plugin is kind of an extension to mysqlnd. Plugins can hook virtually all mysqlnd functions. The mysqlnd functions are called by the PHP MySQL extensions (ext/mysql, ext/mysqli, PDO_MYSQL). Consequently it can be said that a mysqlnd plugin can hook all PHP MySQL userspace functions. Internal mysqlnd function calls cannot only be hooked but also be replaced. There are no limits for manipulating mysqlnd internal function tables: maximum freedom! Drupal, phpMyFAQ, phpMyAdmin, Oxid, … | ext/mysql, ext/mysqli, ext/PDO_MYSQL Mysqlnd Mysqlnd plugin Load Balancing Monitoring Performance | MySQL Server Plugins operate on the C level inside PHP and mysqlnd. They can be made 100% transparent to PHP applications. No application changes are needed because plugins operate on a different layer. A mysqlnd plugin adds a new layer to your setup. However, the new layer is part of a software you already deploy: PHP! A mysqlnd plugin is just another PHP extension in your existing PHP infrastructure. Load Balancing Read/Write Splitting Failover Round-Robin, least loaded Monitoring Query Logging Query Analysis Query Auditing Performance Caching Throttling Sharding Mysqlnd plugins are a different technology than MySQL Proxy. Both are valid tools for solving a variety of common tasks ranging from Load Balancing over Monitoring to Performance. Both have their room, their disadvantages and advantages. An obvious difference: MySQL Proxy works with all MySQL Clients whereas mysqlnd plugins are specific and limited to PHP. mysqlnd plugin vs. MySQL Proxy: architecture A mysqlnd plugin gets installed on the PHP application server. MySQL Proxy can either be run on the PHP applications server or be installed on a dedicated machine to handle multple PHP application servers. Deploying a proxy layer on the application machines has two advantages: no single point of failure easy to scale out (horizontal scale out, scale by client) MySQL Proxy is a wonderful and unique piece of software. MySQL Proxy (and mysqlnd plugins) can solve problems easily which otherwise would have required massive changes to existing applications. But MySQL Proxy comes at a price: MySQL Proxy is a new component and technology to master and deploy MySQL Proxy Lua scripts are not PHP: Lua is a new additional language MySQL Proxy can be customized with C and Lua programming. Lua is the preferred scripting language of MySQL Proxy. For most PHP experts Lua is a new language to learn. A mysqlnd plugin can be written in C (or PHP - as will be shown in future blog posts). C and PHP should be a natural fit for your existing man power and teams. Hardware Software Application server PHP application C/Java/PHP/… application mysqlnd plugin MySQL Proxy   Dedicated machine     MySQL Proxy   Database server MySQL Lifecycle: deamon vs. PHP lifecycle. MySQL Proxy is a deamon. It runs forever. MySQL Proxy can recall earlier decisions. A mysqlnd plugin is bound to the PHP lifecycle of one or multiple web requests. MySQL Proxy can share once computed results among multiple application server. To do the same a mysqlnd plugin needs to store its knowledge in a persistent medium, for example, using another daemon such as Memcache. MySQL Proxy has the edge. mysqlnd plugin: choose C API or wire protocol MySQL Proxy works on top of the wire protocol. With MySQL Proxy you have to parse and reverse engineer the MySQL Client Server Protocol. Actions are limited to what can be done by manipulating the communication protocol. If the wire protocol changes, which happens very rarely, MySQL Proxy scripts need to be changed as well. Layer Software PHP application C/Java/PHP/… application C API mysqlnd plugin Wire protocol mysqlnd plugin MySQL Proxy Mysqlnd plugins work on top of the C API (and thus also on top of the wire protocol). You can hook all C API calls. PHP makes use of the C API. Therefore you can hook all PHP calls. There is no need to go down to the level of the wire protocol. Mysqlnd implements the wire protocol. Plugins can parse, reverse engineer,manipulate and even replace the communication protocol. However, there are few use cases which require you to work on this low level. b>Mysqlnd plugins usually operate the C API layer but they can, if need be, operate on the wire protocol as well.. Plugins let you do your manipulations on two layers: C API and wire protocol. This is more than what MySQL Proxy has to offer. Wire protocol changes do not require plugin changes. Get the mysqlnd plugin API today! If you have recently downloaded a PHP 5.3 development code snapshot you already downloaded the "mysqlnd plugin API". No, there is no downloadable mysqlnd plugin which gives you all the fabled features sketched above. The mysqlnd plugin API is under development since at least christmas 2009. It is developed in the PHP source repository and available to public. The "mysqlnd plugin API" has its roots in the ancient history of mysqlnd. It is a bit of a side effect of Andrey’s attempt to modularize the more than 15k loc of mysqlnd. It took us some time to realize the potential of Andrey’s changes. But this is another story, this is for another blog posting. Like it? More blogging to come Several companies have expressed severe interest in the mysqlnd plugin idea after todays presentation at the IPC Spring. If you see some potential in the new toy, please drop us a note. You can reach Andrey, me and Johannes through the usual channels. More blog postings will follow describing more and more aspects of the new toy. Ideally we manage to teach you how to hack your own mysqlnd plugins - start your GCC’s boys! Background: mysqlnd is a libmysql replacement The MySQL native driver for PHP (mysqlnd) is a C library which implements the MySQL Client Server Protocol. It serves as a drop-in replacement for the MySQL Client Library (AKA libmysql AKA libmysqlclient AKA Connector/C). mysqlnd is also a special kind of a PHP extension. Similar to ext/PDO it does not export any userland functions. It serves as a C library for other extensions. mysqlnd is part of the PHP source code repository as of PHP 5.3. Every PHP source code distribution contains it. Server API (SAPI) CGI CLI Embed ISAPI NSAPI phttpd thttpd … Zend Engine PHP Runtime PHP Extensions bcmath mysql mysqli mysqlnd pdo pdo_mysql xml … All PHP-MySQL APIs can either make use of the MySQL Client Library or mysqlnd to connect to MySQL. The decision to use one or the other library is made at compile time. Within a PHP binary you can mix mysqlnd and the MySQL Client Library as you like: one PHP MySQL API may use mysqlnd and another PHP MySQL extension may use the MySQL Client Library. To use the MySQL Client Library, you must have it installed on your system (at least when building PHP), whereas mysqlnd ships with PHP and thus has no pre-requisites. ";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:7:"pubDate";a:1:{i:0;a:5:{s:4:"data";s:31:"Wed, 02 Jun 2010 19:40:39 +0000";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:8:"category";a:2:{i:0;a:5:{s:4:"data";s:21:"PlanetMySQL (english)";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:1;a:5:{s:4:"data";s:19:"PlanetPHP (english)";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}s:40:"http://purl.org/rss/1.0/modules/content/";a:1:{s:7:"encoded";a:1:{i:0;a:5:{s:4:"data";s:13030:"

The mysqlnd plugin API is a well hidden gem of mysqlnd. Mysqlnd plugins operate on a layer between PHP applications and the MySQL server. This is comparable to MySQL Proxy. MySQL Proxy operates on a layer between any MySQL client application, for example, a PHP application and, the MySQL server. Plugins can take over classical MySQL Proxy tasks such as Load Balancing, Monitoring and Performance optimizations. But due to the different architecture and location mysqlnd plugins do not share some common MySQL Proxy annoyances: no single point of failure, no dedicated proxy server to deploy, no new programming language to learn (Lua).

Slides from the IPC Spring conference

This blog posting and today’s presentation at the IPC Spring 2010, a PHP conference in Berlin, are the first public
information on the new toy. Enjoy the slides or continue reading the blog posting.

The PHP mysqlnd plugin talk - plugins an alternative to MySQL Proxy

View more presentations from Ulf Wendel.

What mysqlnd plugins can do!

A mysqlnd plugin is kind of an extension to mysqlnd. Plugins can hook virtually all mysqlnd functions. The mysqlnd functions are called by the PHP MySQL extensions (ext/mysql, ext/mysqli, PDO_MYSQL). Consequently it can be said that a mysqlnd plugin can hook all PHP MySQL userspace functions.

Internal mysqlnd function calls cannot only be hooked but also be replaced. There are no limits for manipulating mysqlnd internal function tables: maximum freedom!

Drupal, phpMyFAQ, phpMyAdmin, Oxid, …
|
ext/mysql, ext/mysqli, ext/PDO_MYSQL
Mysqlnd
Mysqlnd plugin
Load Balancing Monitoring Performance
|
MySQL Server

Plugins operate on the C level inside PHP and mysqlnd. They can be made 100% transparent to PHP applications. No application changes are needed because plugins operate on a different layer. A mysqlnd plugin adds a new layer to your setup. However, the new layer is part of a software you already deploy: PHP! A mysqlnd plugin is just another PHP extension in your existing PHP infrastructure.

Mysqlnd plugins are a different technology than MySQL Proxy. Both are valid tools for solving a variety of common tasks ranging from Load Balancing over Monitoring to Performance. Both have their room, their disadvantages and advantages. An obvious difference: MySQL Proxy works with all MySQL Clients whereas mysqlnd plugins are specific and limited to PHP.

mysqlnd plugin vs. MySQL Proxy: architecture

A mysqlnd plugin gets installed on the PHP application server. MySQL Proxy can either be run on the PHP applications server or be installed on a dedicated machine to handle multple PHP application servers.

Deploying a proxy layer on the application machines has two advantages:

MySQL Proxy is a wonderful and unique piece of software. MySQL Proxy (and mysqlnd plugins) can solve problems easily which otherwise would have required massive changes to existing applications. But MySQL Proxy comes at a price:

MySQL Proxy can be customized with C and Lua programming. Lua is the preferred scripting language of MySQL Proxy. For most PHP experts Lua is a new language to learn. A mysqlnd plugin can be written in C (or PHP - as will be shown in future blog posts). C and PHP should be a natural fit for your existing man power and teams.

Hardware Software
Application server PHP application C/Java/PHP/… application
mysqlnd plugin MySQL Proxy  
Dedicated machine     MySQL Proxy  
Database server MySQL

Lifecycle: deamon vs. PHP lifecycle. MySQL Proxy is a deamon. It runs forever. MySQL Proxy can recall earlier decisions. A mysqlnd plugin is bound to the PHP lifecycle of one or multiple web requests. MySQL Proxy can share once computed results among multiple application server. To do the same a mysqlnd plugin needs to store its knowledge in a persistent medium, for example, using another daemon such as Memcache. MySQL Proxy has the edge.

mysqlnd plugin: choose C API or wire protocol

MySQL Proxy works on top of the wire protocol. With MySQL Proxy you have to parse and reverse engineer the MySQL Client Server Protocol. Actions are limited to what can be done by manipulating the communication protocol. If the wire protocol changes, which happens very rarely, MySQL Proxy scripts need to be changed as well.

Layer Software
PHP application C/Java/PHP/… application
C API mysqlnd plugin
Wire protocol mysqlnd plugin MySQL Proxy

Mysqlnd plugins work on top of the C API (and thus also on top of the wire protocol). You can hook all C API calls. PHP makes use of the C API. Therefore you can hook all PHP calls. There is no need to go down to the level of the wire protocol.

Mysqlnd implements the wire protocol. Plugins can parse, reverse engineer,manipulate and even replace the communication protocol. However, there are few use cases which require you to work on this low level.

b>Mysqlnd plugins usually operate the C API layer but they can, if need be, operate on the wire protocol as well..

Plugins let you do your manipulations on two layers: C API and wire protocol. This is more than what MySQL Proxy has to offer. Wire protocol changes do not require plugin changes.

Get the mysqlnd plugin API today!

If you have recently downloaded a PHP 5.3 development code snapshot you already downloaded the "mysqlnd plugin API". No, there is no downloadable mysqlnd plugin which gives you all the fabled features sketched above. The mysqlnd plugin API is under development since at least christmas 2009. It is developed in the PHP source repository and available to public.

The "mysqlnd plugin API" has its roots in the ancient history of mysqlnd. It is a bit of a side effect of Andrey’s attempt to modularize the more than 15k loc of mysqlnd. It took us some time to realize the potential of Andrey’s changes. But this is another story, this is for another blog posting.

The mysqlnd plugin talk at the IPC Spring 2010

Like it? More blogging to come

Several companies have expressed severe interest in the mysqlnd plugin idea after todays presentation at the IPC Spring. If you see some potential in the new toy, please drop us a note. You can reach Andrey, me and Johannes through the usual channels.

More blog postings will follow describing more and more aspects of the new toy. Ideally we manage to teach you how to hack your own mysqlnd plugins - start your GCC’s boys!

Background: mysqlnd is a libmysql replacement

The MySQL native driver for PHP (mysqlnd) is a C library which implements the MySQL Client Server Protocol. It serves as a drop-in replacement for the MySQL Client Library (AKA libmysql AKA libmysqlclient AKA Connector/C). mysqlnd is also a special kind of a PHP extension. Similar to ext/PDO it does not export any userland functions. It serves as a C library for other extensions.

mysqlnd is part of the PHP source code repository as of PHP 5.3. Every PHP source code distribution contains it.

Server API (SAPI)
CGI CLI Embed ISAPI NSAPI phttpd thttpd …
Zend Engine PHP Runtime
PHP Extensions
bcmath mysql mysqli mysqlnd pdo pdo_mysql xml …

All PHP-MySQL APIs can either make use of the MySQL Client Library or mysqlnd to connect to MySQL. The decision to use one or the other library is made at compile time. Within a PHP binary you can mix mysqlnd and the MySQL Client Library as you like: one PHP MySQL API may use mysqlnd and another PHP MySQL extension may use the MySQL Client Library.

To use the MySQL Client Library, you must have it installed on your system (at least when building PHP), whereas mysqlnd ships with PHP and thus has no pre-requisites.


PlanetMySQL Voting: Vote UP / Vote DOWN";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}s:32:"http://purl.org/dc/elements/1.1/";a:1:{s:7:"creator";a:1:{i:0;a:5:{s:4:"data";s:10:"Ulf Wendel";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}}}i:41;a:6:{s:4:"data";s:48:" ";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";s:5:"child";a:3:{s:0:"";a:6:{s:5:"title";a:1:{i:0;a:5:{s:4:"data";s:36:"Security fixes for MySQL 4.0 and 4.1";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:4:"guid";a:1:{i:0;a:5:{s:4:"data";s:43:"http://www.mysqlperformanceblog.com/?p=2918";s:7:"attribs";a:1:{s:0:"";a:1:{s:11:"isPermaLink";s:5:"false";}}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:4:"link";a:1:{i:0;a:5:{s:4:"data";s:84:"http://www.mysqlperformanceblog.com/2010/06/02/security-fixes-for-mysql-4-0-and-4-1/";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:11:"description";a:1:{i:0;a:5:{s:4:"data";s:823:"In Percona Server security fix releases I mentioned patches for MySQL 4.0 and 4.1. I am happy to announce that GoDaddy.com released patches for MySQL 4.0 and MySQL 4.1 under GPL license and you can get them from our Launchpad: for 4.0: lp:~percona-dev/percona-patches/4.0.30 ( or https://launchpad.net/~percona-dev/percona-patches/4.0.30 ) for 4.1: lp:~percona-dev/percona-patches/4.1.24 ( or https://launchpad.net/~percona-dev/percona-patches/4.1.24) Fixed bugs: Bug#53371, CVE-2010-1848 Bug#53237, CVE-2010-1850 Bug#50974, CVE-2010-1849 If you are using MySQL 4.0 and 4.1 with public user access we strongly recommend to patch your MySQL instances. You need help with binaries you may contact us or wait until we have binaries public available. Entry posted by Vadim | No comment Add to: | | | | ";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:7:"pubDate";a:1:{i:0;a:5:{s:4:"data";s:31:"Wed, 02 Jun 2010 19:16:01 +0000";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:8:"category";a:2:{i:0;a:5:{s:4:"data";s:5:"mysql";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:1;a:5:{s:4:"data";s:5:"patch";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}s:40:"http://purl.org/rss/1.0/modules/content/";a:1:{s:7:"encoded";a:1:{i:0;a:5:{s:4:"data";s:3839:"

In Percona Server security fix releases I mentioned patches for MySQL 4.0 and 4.1.

I am happy to announce that GoDaddy.com released patches for MySQL 4.0 and MySQL 4.1 under GPL license and you can get them from our Launchpad:

for 4.0: lp:~percona-dev/percona-patches/4.0.30 ( or https://launchpad.net/~percona-dev/percona-patches/4.0.30 )
for 4.1: lp:~percona-dev/percona-patches/4.1.24 ( or https://launchpad.net/~percona-dev/percona-patches/4.1.24)

Fixed bugs:

If you are using MySQL 4.0 and 4.1 with public user access we strongly recommend to patch your MySQL instances. You need help with binaries you may contact us or wait until we have binaries public available.

Entry posted by Vadim | No comment

Add to: delicious | digg | reddit | netscape | Google Bookmarks


PlanetMySQL Voting: Vote UP / Vote DOWN";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}s:32:"http://purl.org/dc/elements/1.1/";a:1:{s:7:"creator";a:1:{i:0;a:5:{s:4:"data";s:22:"MySQL Performance Blog";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}}}i:42;a:6:{s:4:"data";s:38:" ";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";s:5:"child";a:3:{s:0:"";a:5:{s:5:"title";a:1:{i:0;a:5:{s:4:"data";s:39:"Finding memleaks in win32 with valgrind";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:4:"guid";a:1:{i:0;a:5:{s:4:"data";s:34:"tag:jan.kneschke.de,2010-06-02:570";s:7:"attribs";a:1:{s:0:"";a:1:{s:11:"isPermaLink";s:5:"false";}}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:4:"link";a:1:{i:0;a:5:{s:4:"data";s:71:"http://jan.kneschke.de/2010/6/2/finding-memleaks-in-win32-with-valgrind";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:11:"description";a:1:{i:0;a:5:{s:4:"data";s:2397:"I use MacOS X for most of my development work, but part of my job is writing apps that run on all major Unixes and Win32. Most of the time bugs and memleaks are "portable", meaning that they appear on all platforms in the same way and fixing them on your dev-platforms, fixes them everywhere else. Well, "most of the time". Today I had the joy of a win32 only mem-leak. We actually ''bisect''ed it down to the locking support for openssl that we just added http://www.openssl.org/docs/crypto/threads.html. But starring at the code didn't reveal anything obvious. So, back to the tools. Let them find it. Going through the web on the search for "how to find mem-leaks on win32" I came along: http://wiki.winehq.org/Wine_and_Valgrind The idea is simple: use wine to run the win32 app run wine in valgrind You get all the benefits of valgrind and can work on the OS you are used to. But not so fast: wine from MacPorts is 32bit only http://trac.macports.org/browser/trunk/dports/x11/wine-devel/Portfile ... but all your macports (at least on Mac OS X 10.6) dependencies are 64bit only. After rebuilding the dependencies with +universal, wine starts up nicely: [jan@macbook bin]$ file mysql-proxy.exe mysql-proxy.exe: PE32 executable for MS Windows (console) Intel 80386 32-bit [jan@macbook bin]$ wine mysql-proxy.exe --version mysql-proxy 0.8.0 chassis: mysql-proxy 0.8.0 glib2: 2.20.3 libevent: 1.4.12-stable LUA: Lua 5.1.2 package.path: Z:\Users\jan\projects\in-targz\mysql-monitor-agent-2.2.0.1717-win32-x86\lib package.cpath: Z:\Users\jan\projects\in-targz\mysql-monitor-agent-2.2.0.1717-win32-x86\bin\lua-?.dll -- modules admin: 0.7.0 proxy: 0.7.0 valgrind fails to build as +universal if built from MacPorts http://trac.macports.org/browser/trunk/dports/devel/valgrind-devel/Portfile ... and ends up as 64bit app ... which can't run a 32bit 'wine'. Back to "build valgrind by hand". Using the Portfile as template I set up the valgrind source tree and built it with '--enable-32bitonly' to get a 32bit valgrind. Everything in place that I need I did hit a bug in valgrind: https://bugs.kde.org/show_bug.cgi?id=194757 Anyway, some real hints how to track down the mem-leaks on win32: http://mcfunley.com/277/using-leakdiag-to-debug-unmanaged-memory-leaks/comment-page-1 http://thetweaker.wordpress.com/2009/04/09/native-memory-leaks-part-1-leakdiag/ ";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:7:"pubDate";a:1:{i:0;a:5:{s:4:"data";s:31:"Wed, 02 Jun 2010 18:45:00 +0000";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}s:40:"http://purl.org/rss/1.0/modules/content/";a:1:{s:7:"encoded";a:1:{i:0;a:5:{s:4:"data";s:3355:"

I use MacOS X for most of my development work, but part of my job is writing apps that run on all major Unixes and Win32. Most of the time bugs and memleaks are "portable", meaning that they appear on all platforms in the same way and fixing them on your dev-platforms, fixes them everywhere else.

Well, "most of the time".

Today I had the joy of a win32 only mem-leak. We actually ''bisect''ed it down to the locking support for openssl that we just added http://www.openssl.org/docs/crypto/threads.html. But starring at the code didn't reveal anything obvious. So, back to the tools. Let them find it.

Going through the web on the search for "how to find mem-leaks on win32" I came along:

The idea is simple:

You get all the benefits of valgrind and can work on the OS you are used to. But not so fast:

... but all your macports (at least on Mac OS X 10.6) dependencies are 64bit only. After rebuilding the dependencies with +universal, wine starts up nicely:

[jan@macbook bin]$ file mysql-proxy.exe 
mysql-proxy.exe: PE32 executable for MS Windows (console) Intel 80386 32-bit
[jan@macbook bin]$ wine mysql-proxy.exe --version
mysql-proxy 0.8.0
  chassis: mysql-proxy 0.8.0
  glib2: 2.20.3
  libevent: 1.4.12-stable
  LUA: Lua 5.1.2
    package.path: Z:\Users\jan\projects\in-targz\mysql-monitor-agent-2.2.0.1717-win32-x86\lib
    package.cpath: Z:\Users\jan\projects\in-targz\mysql-monitor-agent-2.2.0.1717-win32-x86\bin\lua-?.dll
-- modules
  admin: 0.7.0
  proxy: 0.7.0

... and ends up as 64bit app ... which can't run a 32bit 'wine'. Back to "build valgrind by hand". Using the Portfile as template I set up the valgrind source tree and built it with '--enable-32bitonly' to get a 32bit valgrind.

Everything in place that I need I did hit a bug in valgrind: https://bugs.kde.org/show_bug.cgi?id=194757

Anyway, some real hints how to track down the mem-leaks on win32:


PlanetMySQL Voting: Vote UP / Vote DOWN";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}s:32:"http://purl.org/dc/elements/1.1/";a:1:{s:7:"creator";a:1:{i:0;a:5:{s:4:"data";s:12:"Jan Kneschke";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}}}i:43;a:6:{s:4:"data";s:68:" ";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";s:5:"child";a:3:{s:0:"";a:6:{s:5:"title";a:1:{i:0;a:5:{s:4:"data";s:61:"Disk seeks are evil, so let’s avoid them, pt. 3 (Deletions)";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:4:"guid";a:1:{i:0;a:5:{s:4:"data";s:26:"http://tokutek.com/?p=1488";s:7:"attribs";a:1:{s:0:"";a:1:{s:11:"isPermaLink";s:5:"false";}}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:4:"link";a:1:{i:0;a:5:{s:4:"data";s:90:"http://tokutek.com/2010/06/disk-seeks-are-evil-so-let%E2%80%99s-avoid-them-pt-3-deletions/";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:11:"description";a:1:{i:0;a:5:{s:4:"data";s:3250:" As mentioned in parts 1 and 2, having many disk seeks are bad (they slow down performance). Fractal tree data structures minimize disk seeks on ad-hoc insertions, whereas B-trees practically guarantee that disk seeks are performed on ad-hoc insertions. As a result, fractal tree data structures can insert data up to two orders of magnitude faster than B-Trees can. In this post, let’s examine deletions, and get an intuitive understanding for why fractal-tree data structures exhibit the same two orders of magnitude faster deletions than B-trees. In MySQL 5.1, this advantage is really eye-popping for TokuDB v. InnoDB, because InnoDB does not use its insert buffer for deletions. I understand there is a delete buffer in 5.5, which I haven’t experimented with yet. B-trees exhibit the same weakness on deletions as they do on insertions: they need to have the appropriate leaf node in memory. For large tables, bringing the leaf node into memory often requires a disk seek. Fractal tree data structures do not have this requirement. Before going on, a clarification. In MySQL, delete statements have two steps: queries and value changes. For instance, the statement: delete from foo where a=1; must first query all rows where a=1 (the first step), and then proceed to remove the rows that are found (the second step). In this post, we focus on the second step. For storage engine developers, this is the function handler::delete_row. In a future post, I will analyze the first step, tie it together with this post, and show how deletions can be fast in MySQL with TokuDB. Back to deletions. Let’s analyze value changes. We know the contents of the row being deleted. So how can fractal tree data structures avoid an unnecessary disk seek? The answer: deletion messages (sometimes called tombstone deletes). Suppose we have a fractal tree data structure with the following elements inserted: (1), (3),…(999). Up until now, we shown the fractal tree as such. - - - - - - - ... 1 3 5 7 9 ... 999 In reality, the elements stored are not just keys, but rather (message, key) pairs. The message may be one of two operations: insertion or deletion. We represent an insertion with the message ‘i’. So, the fractal tree looks more like this: - - - - - - - ... (i,1) (i,3) (i,5)... (i,999) To delete an element, for example (5), we insert a deletion message into the tree, marking it with a ‘d’. So, after deleting (5), the fractal tree data structure looks like this: (d,5) - - - - - - ... (i,1) (i,3) (i,5)... (i,999) With this scheme, deletions are as fast as insertions, which is to say two orders of magnitude faster than insertions or deletions into a B-tree. On queries, a message in a higher node overrides messages in lower nodes. So upon querying (5), a cursor notices that (d,5) is located higher than (i,5), and therefore the key (5) does not exist in the fractal tree data structure. On merges, the deletion message and insertion message cancel each other out, and space is reclaimed. So, by using deletion messages and treating deletions like insertions, fractal tree data structures can achieve the same performance boost (two orders of magnitude) over B-trees.";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:7:"pubDate";a:1:{i:0;a:5:{s:4:"data";s:31:"Wed, 02 Jun 2010 17:40:26 +0000";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:8:"category";a:6:{i:0;a:5:{s:4:"data";s:8:"TokuView";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:1;a:5:{s:4:"data";s:6:"B-tree";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:2;a:5:{s:4:"data";s:9:"disk seek";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:3;a:5:{s:4:"data";s:13:"Fractal Trees";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:4;a:5:{s:4:"data";s:5:"mysql";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:5;a:5:{s:4:"data";s:6:"TokuDB";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}s:40:"http://purl.org/rss/1.0/modules/content/";a:1:{s:7:"encoded";a:1:{i:0;a:5:{s:4:"data";s:3765:"

As mentioned in parts 1 and 2, having many disk seeks are bad (they slow down performance). Fractal tree data structures minimize disk seeks on ad-hoc insertions, whereas B-trees practically guarantee that disk seeks are performed on ad-hoc insertions. As a result, fractal tree data structures can insert data up to two orders of magnitude faster than B-Trees can.

In this post, let’s examine deletions, and get an intuitive understanding for why fractal-tree data structures exhibit the same two orders of magnitude faster deletions than B-trees. In MySQL 5.1, this advantage is really eye-popping for TokuDB v. InnoDB, because InnoDB does not use its insert buffer for deletions. I understand there is a delete buffer in 5.5, which I haven’t experimented with yet.

B-trees exhibit the same weakness on deletions as they do on insertions: they need to have the appropriate leaf node in memory. For large tables, bringing the leaf node into memory often requires a disk seek. Fractal tree data structures do not have this requirement.

Before going on, a clarification. In MySQL, delete statements have two steps: queries and value changes. For instance, the statement:

delete from foo where a=1;

must first query all rows where a=1 (the first step), and then proceed to remove the rows that are found (the second step). In this post, we focus on the second step. For storage engine developers, this is the function handler::delete_row. In a future post, I will analyze the first step, tie it together with this post, and show how deletions can be fast in MySQL with TokuDB.

Back to deletions. Let’s analyze value changes. We know the contents of the row being deleted. So how can fractal tree data structures avoid an unnecessary disk seek? The answer: deletion messages (sometimes called tombstone deletes).

Suppose we have a fractal tree data structure with the following elements inserted: (1), (3),…(999). Up until now, we shown the fractal tree as such.

-

- -

- - - -

...

1 3 5 7 9 ... 999

In reality, the elements stored are not just keys, but rather (message, key) pairs. The message may be one of two operations: insertion or deletion. We represent an insertion with the message ‘i’. So, the fractal tree looks more like this:

-

- -

- - - -

...

(i,1) (i,3) (i,5)... (i,999)

To delete an element, for example (5), we insert a deletion message into the tree, marking it with a ‘d’. So, after deleting (5), the fractal tree data structure looks like this:

(d,5)

- -

- - - -

...

(i,1) (i,3) (i,5)... (i,999)

With this scheme, deletions are as fast as insertions, which is to say two orders of magnitude faster than insertions or deletions into a B-tree.

On queries, a message in a higher node overrides messages in lower nodes. So upon querying (5), a cursor notices that (d,5) is located higher than (i,5), and therefore the key (5) does not exist in the fractal tree data structure. On merges, the deletion message and insertion message cancel each other out, and space is reclaimed.

So, by using deletion messages and treating deletions like insertions, fractal tree data structures can achieve the same performance boost (two orders of magnitude) over B-trees.


PlanetMySQL Voting: Vote UP / Vote DOWN";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}s:32:"http://purl.org/dc/elements/1.1/";a:1:{s:7:"creator";a:1:{i:0;a:5:{s:4:"data";s:13:"Tokuview Blog";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}}}i:44;a:6:{s:4:"data";s:68:" ";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";s:5:"child";a:3:{s:0:"";a:6:{s:5:"title";a:1:{i:0;a:5:{s:4:"data";s:51:"MySQL University: New Features in Connector/NET 6.3";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:4:"guid";a:1:{i:0;a:5:{s:4:"data";s:67:"http://blogs.sun.com/mysqlf/entry/mysql_university_new_features_in1";s:7:"attribs";a:1:{s:0:"";a:1:{s:11:"isPermaLink";s:5:"false";}}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:4:"link";a:1:{i:0;a:5:{s:4:"data";s:67:"http://blogs.sun.com/mysqlf/entry/mysql_university_new_features_in1";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:11:"description";a:1:{i:0;a:5:{s:4:"data";s:790:"This Thursday (June 3rd, 14:00 UTC), Reggie Burnett, head of Connector/NET development, will present the New Features in Connector/NET 6.3. For MySQL University sessions, point your browser to this page. You need a browser with a working Flash plugin. You may register for a Dimdim account, but you don't have to. (Dimdim is the conferencing system we're using for MySQL University sessions. It provides integrated voice streaming, chat, whiteboard, session recording, and more.) MySQL University is a free educational online program for engineers/developers. MySQL University sessions are open to anyone. All sessions (slides & audio) are recorded; the links will be on the respective MySQL University session pages which are listed on the MySQL University home page.";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:7:"pubDate";a:1:{i:0;a:5:{s:4:"data";s:31:"Wed, 02 Jun 2010 09:03:05 +0000";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:8:"category";a:6:{i:0;a:5:{s:4:"data";s:3:"Sun";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:1;a:5:{s:4:"data";s:9:"connector";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:2;a:5:{s:4:"data";s:13:"connector/net";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:3;a:5:{s:4:"data";s:10:"connectors";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:4;a:5:{s:4:"data";s:5:"mysql";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:5;a:5:{s:4:"data";s:10:"university";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}s:40:"http://purl.org/rss/1.0/modules/content/";a:1:{s:7:"encoded";a:1:{i:0;a:5:{s:4:"data";s:1450:"

This Thursday (June 3rd, 14:00 UTC), Reggie Burnett, head of Connector/NET development, will present the New Features in Connector/NET 6.3.

For MySQL University sessions, point your browser to this page. You need a browser with a working Flash plugin. You may register for a Dimdim account, but you don't have to. (Dimdim is the conferencing system we're using for MySQL University sessions. It provides integrated voice streaming, chat, whiteboard, session recording, and more.)

MySQL University is a free educational online program for engineers/developers. MySQL University sessions are open to anyone. All sessions (slides & audio) are recorded; the links will be on the respective MySQL University session pages which are listed on the MySQL University home page.


PlanetMySQL Voting: Vote UP / Vote DOWN";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}s:32:"http://purl.org/dc/elements/1.1/";a:1:{s:7:"creator";a:1:{i:0;a:5:{s:4:"data";s:11:"Stefan Hinz";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}}}i:45;a:6:{s:4:"data";s:48:" ";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";s:5:"child";a:3:{s:0:"";a:6:{s:5:"title";a:1:{i:0;a:5:{s:4:"data";s:35:"Color code your performance numbers";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:4:"guid";a:1:{i:0;a:5:{s:4:"data";s:43:"http://www.mysqlperformanceblog.com/?p=2914";s:7:"attribs";a:1:{s:0:"";a:1:{s:11:"isPermaLink";s:5:"false";}}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:4:"link";a:1:{i:0;a:5:{s:4:"data";s:83:"http://www.mysqlperformanceblog.com/2010/06/02/color-code-your-performance-numbers/";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:11:"description";a:1:{i:0;a:5:{s:4:"data";s:6545:"When analyzing how good or bad response time is it is not handy to look at the averages, min or max times - something what is easily computed using built in aggregate functions. We most likely would like to see some percentile numbers - 95 percentile or 99 percentile. The problem is computing these in SQL is ugly and resource consuming. There is however a simple way to get similar data, looking at it from the different point of view. When we're speaking about application we may not always care about exact value of response time but rather we want to see response time to be within certain range. For example if we define page feels good if response time is below 50ms it is not as important if response was 40ms or 44ms - it is much more important how frequently this goal was reached. In fact I prefer to define 2 performance level. One what users would consider good performance, another what is acceptable, and basically the third level is unacceptable. We can also set classical color codes to them - green, yellow, red. For example we can define good performance as response time for search pages within 1 second, acceptable as response within 3 seconds and anything over that will be considered unacceptable: PLAIN TEXT SQL: mysql> SELECT sum(wtime<1.0)/count(*)*100 green_pct, sum(wtime>=1.0 AND wtime<3.0)/count(*)*100 yellow_pct, sum(wtime>=3.)/count(*)*100 red_pct FROM performance_log_100601 WHERE page_type='search'; +-----------+------------+---------+ | green_pct | yellow_pct | red_pct | +-----------+------------+---------+ |   99.6701 |     0.3006 |  0.0293 | +-----------+------------+---------+ 1 row IN SET (24.87 sec) These are great numbers to look after. In this case we can say system has "two nines" in terms of performance - it responds with what we consider "good" performance in over 99% cases. I consider this a lot more meaningful number for business than speaking about 99% response time, especially as it also clearly clarifies how bad the problem is. If your goal is 99% response time is 1sec and you're seeing it 2 seconds what you really do not know is how many people fall in this 1-sec to 2 sec range. It could be very small number and so your performance is not that bad or it could be majority of response times are withing 1 to 2 sec range. Next thing I would add is the average response time for our good requests. This eliminates slow outliers which we just count by their number and allows to see how good our performance really is. I consider it secondary value as our major goal is eliminating variance as defined by Six Sigma practices. However given variance is the same better average response time for good requests helps us to track performance changes closely and act before we get large number of yellow or red requests: PLAIN TEXT SQL: mysql> SELECT avg(IF(wtime<1.0,wtime,NULL)) avg_green,sum(wtime<1.0)/count(*)*100 green_pct, sum(wtime>=1.0 AND wtime<3.0)/count(*)*100 yellow_pct, sum(wtime>=3.0)/count(*)*100 red_pct FROM performance_log_100601 WHERE page_type='search'; +-------------------+-----------+------------+---------+ | avg_green         | green_pct | yellow_pct | red_pct | +-------------------+-----------+------------+---------+ | 0.448726461517477 |   99.6701 |     0.3006 |  0.0293 | +-------------------+-----------+------------+---------+ 1 row IN SET (6.84 sec) Finally I find it helpful to track it among time period. For example in this case we're looking for data grouped by 15 min interval though you may be looking at different intervals too: PLAIN TEXT SQL: mysql> SELECT from_unixtime(unix_timestamp(logged) div 900 * 900) period, avg(IF(wtime<1.0,wtime,NULL)) avg_green,sum(wtime<1.0)/count(*)*100 green_pct, sum(wtime>=1.0 AND wtime<3.0)/count(*)*100 yellow_pct, sum(wtime>=3.0)/count(*)*100 red_pct FROM performance_log_100601 WHERE page_type='search' GROUP BY 1; +---------------------+-------------------+-----------+------------+---------+ | period              | avg_green         | green_pct | yellow_pct | red_pct | +---------------------+-------------------+-----------+------------+---------+ | 2010-05-31 20:00:00 | 0.454738099705166 |  100.0000 |     0.0000 |  0.0000 | | 2010-05-31 20:15:00 | 0.477789820622135 |  100.0000 |     0.0000 |  0.0000 | | 2010-05-31 20:30:00 | 0.463134796179216 |  100.0000 |     0.0000 |  0.0000 | | 2010-05-31 20:45:00 | 0.397884420995359 |   99.4595 |     0.5405 |  0.0000 | | 2010-05-31 21:00:00 | 0.438939880065877 |  100.0000 |     0.0000 |  0.0000 | | 2010-05-31 21:15:00 |  0.48698966072074 |  100.0000 |     0.0000 |  0.0000 | | 2010-05-31 21:30:00 | 0.432073892663709 |   93.6364 |     6.3636 |  0.0000 | | 2010-05-31 21:45:00 | 0.483174517098134 |  100.0000 |     0.0000 |  0.0000 | | 2010-05-31 22:00:00 | 0.449507013872618 |  100.0000 |     0.0000 |  0.0000 | | 2010-05-31 22:15:00 | 0.327130372130607 |  100.0000 |     0.0000 |  0.0000 | | 2010-05-31 22:30:00 | 0.431361319540545 |  100.0000 |     0.0000 |  0.0000 | | 2010-05-31 22:45:00 | 0.480885540467242 |  100.0000 |     0.0000 |  0.0000 | Note response time does not always correlates with number of bad requests. For example we can see at one interval there were over 6% requests in yellow but average response time for good requests actually was better than in the other intervals. The queries which produce this data may not be overly beautiful but they are pretty fast requiring simple scan of the data range. P.S Gathering data this way also often gives you some idea about what your percentile response time numbers might be. In this case I know over 99% of requests are served in 1sec so I know my 99% response time is below 1 second. Entry posted by peter | 4 comments Add to: | | | | ";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:7:"pubDate";a:1:{i:0;a:5:{s:4:"data";s:31:"Wed, 02 Jun 2010 07:11:26 +0000";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:8:"category";a:2:{i:0;a:5:{s:4:"data";s:5:"mysql";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:1;a:5:{s:4:"data";s:10:"production";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}s:40:"http://purl.org/rss/1.0/modules/content/";a:1:{s:7:"encoded";a:1:{i:0;a:5:{s:4:"data";s:13813:"

When analyzing how good or bad response time is it is not handy to look at the averages, min or max times - something what is easily computed using built in aggregate functions. We most likely would like to see some percentile numbers - 95 percentile or 99 percentile. The problem is computing these in SQL is ugly and resource consuming. There is however a simple way to get similar data, looking at it from the different point of view.

When we're speaking about application we may not always care about exact value of response time but rather we want to see response time to be within certain range. For example if we define page feels good if response time is below 50ms it is not as important if response was 40ms or 44ms - it is much more important how frequently this goal was reached.

In fact I prefer to define 2 performance level. One what users would consider good performance, another what is acceptable, and basically the third level is unacceptable. We can also set classical color codes to them - green, yellow, red.

For example we can define good performance as response time for search pages within 1 second, acceptable as response within 3 seconds and anything over that will be considered unacceptable:

PLAIN TEXT
SQL:
  1. mysql> SELECT sum(wtime<1.0)/count(*)*100 green_pct, sum(wtime>=1.0 AND wtime<3.0)/count(*)*100 yellow_pct, sum(wtime>=3.)/count(*)*100 red_pct FROM performance_log_100601 WHERE page_type='search';
  2. +-----------+------------+---------+
  3. | green_pct | yellow_pct | red_pct |
  4. +-----------+------------+---------+
  5. |   99.6701 |     0.3006 |  0.0293 |
  6. +-----------+------------+---------+
  7. 1 row IN SET (24.87 sec)

These are great numbers to look after. In this case we can say system has "two nines" in terms of performance - it responds with what we consider "good" performance in over 99% cases. I consider this a lot more meaningful number for business than speaking about 99% response time, especially as it also clearly clarifies how bad the problem is. If your goal is 99% response time is 1sec and you're seeing it 2 seconds what you really do not know is how many people fall in this 1-sec to 2 sec range. It could be very small number and so your performance is not that bad or it could be majority of response times are withing 1 to 2 sec range.

Next thing I would add is the average response time for our good requests. This eliminates slow outliers which we just count by their number and allows to see how good our performance really is. I consider it secondary value as our major goal is eliminating variance as defined by Six Sigma practices. However given variance is the same better average response time for good requests helps us to track performance changes closely and act before we get large number of yellow or red requests:

PLAIN TEXT
SQL:
  1. mysql> SELECT avg(IF(wtime<1.0,wtime,NULL)) avg_green,sum(wtime<1.0)/count(*)*100 green_pct, sum(wtime>=1.0 AND wtime<3.0)/count(*)*100 yellow_pct, sum(wtime>=3.0)/count(*)*100 red_pct FROM performance_log_100601 WHERE page_type='search';
  2. +-------------------+-----------+------------+---------+
  3. | avg_green         | green_pct | yellow_pct | red_pct |
  4. +-------------------+-----------+------------+---------+
  5. | 0.448726461517477 |   99.6701 |     0.3006 |  0.0293 |
  6. +-------------------+-----------+------------+---------+
  7. 1 row IN SET (6.84 sec)

Finally I find it helpful to track it among time period. For example in this case we're looking for data grouped by 15 min interval though you may be looking at different intervals too:

PLAIN TEXT
SQL:
  1. mysql> SELECT from_unixtime(unix_timestamp(logged) div 900 * 900) period, avg(IF(wtime<1.0,wtime,NULL)) avg_green,sum(wtime<1.0)/count(*)*100 green_pct, sum(wtime>=1.0 AND wtime<3.0)/count(*)*100 yellow_pct, sum(wtime>=3.0)/count(*)*100 red_pct FROM performance_log_100601 WHERE page_type='search' GROUP BY 1;
  2. +---------------------+-------------------+-----------+------------+---------+
  3. | period              | avg_green         | green_pct | yellow_pct | red_pct |
  4. +---------------------+-------------------+-----------+------------+---------+
  5. | 2010-05-31 20:00:00 | 0.454738099705166 |  100.0000 |     0.0000 |  0.0000 |
  6. | 2010-05-31 20:15:00 | 0.477789820622135 |  100.0000 |     0.0000 |  0.0000 |
  7. | 2010-05-31 20:30:00 | 0.463134796179216 |  100.0000 |     0.0000 |  0.0000 |
  8. | 2010-05-31 20:45:00 | 0.397884420995359 |   99.4595 |     0.5405 |  0.0000 |
  9. | 2010-05-31 21:00:00 | 0.438939880065877 |  100.0000 |     0.0000 |  0.0000 |
  10. | 2010-05-31 21:15:00 |  0.48698966072074 |  100.0000 |     0.0000 |  0.0000 |
  11. | 2010-05-31 21:30:00 | 0.432073892663709 |   93.6364 |     6.3636 |  0.0000 |
  12. | 2010-05-31 21:45:00 | 0.483174517098134 |  100.0000 |     0.0000 |  0.0000 |
  13. | 2010-05-31 22:00:00 | 0.449507013872618 |  100.0000 |     0.0000 |  0.0000 |
  14. | 2010-05-31 22:15:00 | 0.327130372130607 |  100.0000 |     0.0000 |  0.0000 |
  15. | 2010-05-31 22:30:00 | 0.431361319540545 |  100.0000 |     0.0000 |  0.0000 |
  16. | 2010-05-31 22:45:00 | 0.480885540467242 |  100.0000 |     0.0000 |  0.0000 |

Note response time does not always correlates with number of bad requests. For example we can see at one interval there were over 6% requests in yellow but average response time for good requests actually was better than in the other intervals.

The queries which produce this data may not be overly beautiful but they are pretty fast requiring simple scan of the data range.

P.S Gathering data this way also often gives you some idea about what your percentile response time numbers might be. In this case I know over 99% of requests are served in 1sec so I know my 99% response time is below 1 second.

Entry posted by peter | 4 comments

Add to: delicious | digg | reddit | netscape | Google Bookmarks


PlanetMySQL Voting: Vote UP / Vote DOWN";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}s:32:"http://purl.org/dc/elements/1.1/";a:1:{s:7:"creator";a:1:{i:0;a:5:{s:4:"data";s:22:"MySQL Performance Blog";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}}}i:46;a:6:{s:4:"data";s:78:" ";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";s:5:"child";a:3:{s:0:"";a:6:{s:5:"title";a:1:{i:0;a:5:{s:4:"data";s:42:"Disrupting IT with Open Source & Cloud";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:4:"guid";a:1:{i:0;a:5:{s:4:"data";s:60:"tag:typepad.com,2003:post-6a00d83452e46469e2013482c906b0970c";s:7:"attribs";a:1:{s:0:"";a:1:{s:11:"isPermaLink";s:5:"false";}}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:4:"link";a:1:{i:0;a:5:{s:4:"data";s:54:"http://www.theopenforce.com/2010/06/disrupting-it.html";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:11:"description";a:1:{i:0;a:5:{s:4:"data";s:978:"A couple of weeks ago I gave a presentation at the Apache Lucene Eurocon in Prague. It was a good conference focused on Lucene/Solr open source search technology and sponsored by Lucid Imagination.   I've posted the bulk of the presentation below.  (I omitted a couple of slides that were MySQL specific.) Even though it was a technical conference, I got positive feedback from the attendees and organizers that the information was useful in helping folks think about where to focus their efforts.   The slides have been posted to Box.net and are shown using their new "embedded preview" feature which is pretty cool. You can also use the short URL www.tinyurl.com/box-disr to view or download the slides in Powerpoint format. Thanks to the folks at Lucid Imagination as well as those who gave input and feedback on the presentation. Conference: Apache Lucene Eurocon, Agenda, Training Lucid Imagination: Main site, Blog, Training, Services ";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:7:"pubDate";a:1:{i:0;a:5:{s:4:"data";s:31:"Wed, 02 Jun 2010 04:46:05 +0000";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:8:"category";a:8:{i:0;a:5:{s:4:"data";s:8:"Business";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:1;a:5:{s:4:"data";s:6:"Apache";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:2;a:5:{s:4:"data";s:10:"conference";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:3;a:5:{s:4:"data";s:10:"Disruption";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:4;a:5:{s:4:"data";s:7:"eurocon";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:5;a:5:{s:4:"data";s:6:"Lucene";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:6;a:5:{s:4:"data";s:6:"prague";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:7;a:5:{s:4:"data";s:4:"Solr";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}s:40:"http://purl.org/rss/1.0/modules/content/";a:1:{s:7:"encoded";a:1:{i:0;a:5:{s:4:"data";s:2153:"

A couple of weeks ago I gave a presentation at the Apache Lucene Eurocon in Prague. It was a good conference focused on Lucene/Solr open source search technology and sponsored by Lucid Imagination.  

I've posted the bulk of the presentation below.  (I omitted a couple of slides that were MySQL specific.) Even though it was a technical conference, I got positive feedback from the attendees and organizers that the information was useful in helping folks think about where to focus their efforts.  

The slides have been posted to Box.net and are shown using their new "embedded preview" feature which is pretty cool. You can also use the short URL www.tinyurl.com/box-disr to view or download the slides in Powerpoint format.

Thanks to the folks at Lucid Imagination as well as those who gave input and feedback on the presentation.


PlanetMySQL Voting: Vote UP / Vote DOWN";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}s:32:"http://purl.org/dc/elements/1.1/";a:1:{s:7:"creator";a:1:{i:0;a:5:{s:4:"data";s:13:"Zack Urlocker";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}}}i:47;a:6:{s:4:"data";s:68:" ";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";s:5:"child";a:3:{s:0:"";a:6:{s:5:"title";a:1:{i:0;a:5:{s:4:"data";s:42:"Under-provisioning: the curse of the cloud";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:4:"guid";a:1:{i:0;a:5:{s:4:"data";s:33:"http://www.xaprb.com/blog/?p=1884";s:7:"attribs";a:1:{s:0:"";a:1:{s:11:"isPermaLink";s:5:"false";}}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:4:"link";a:1:{i:0;a:5:{s:4:"data";s:79:"http://www.xaprb.com/blog/2010/06/01/under-provisioning-the-curse-of-the-cloud/";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:11:"description";a:1:{i:0;a:5:{s:4:"data";s:5138:"A common problem I see people running into when using a cloud computing service is the trap of under-provisioning. There’s a chain effect that leads to this result: 1) people don’t understand how virtualization works, and therefore 2) they don’t realize how much of a computing resource they’re really buying, so 3) they assume they are entitled to more than they really are, and 4) they under-provision. A few other causes and effects come into play here, too. For example, the choice to use the cloud is sometimes founded on economic assumptions that frequently turn out to be wrong. The cloud service looks more economically attractive than it really is, due to under-provisioning. Let’s get back to this idea that people under-provision. How do I know that’s happening? I’ll use anecdotal evidence to illustrate. Here’s a real quote from a recent engagement about database (MySQL) performance problems: Do you think it’s likely that the underlying hardware is simply worse than average? If you think this will be an ongoing problem, maybe we should try our luck with a new instance/storage cluster? The fundamental assumption here is that some clusters are overloaded and are giving poor quality of service. We’re trained to think this way because we are familiar with services such as shared hosting, where other users on your particular server might really be abusive and claim resources that should be yours. But this isn’t how virtualization works in the common cloud platforms. In these platforms, you aren’t sharing resources with other users. You are guaranteed to get what you deserve! No kidding — this actually works. If that’s true, then why does performance fluctuate so much? The answer lies in how resources are parceled out. Assume there are 10 units of computing resources, and you’re paying for one of them. You buy 1/10th of the machine’s power. But it just happens that you’re the only virtual instance running on that physical server. You fire up an intense job. How much power do you get? You paid for 1 unit, but you get 10, because no one else is using the other 9 units. This is the way most virtualization platforms work: they give you extra resources if they’re available and not being claimed by anyone else’s instance. This guarantees that you’ll never get less than you deserve, but it leaves open the possibility that you’ll get more than you deserve. (What would be the point of wasting that power, really?) Under-provisioning is the obverse of over-providing, which is what the virtualization platform does. First-generation hyperthreading gave the same illusion of more resources than are really available, by the way. It made you think there were multiple processors, when in fact there weren’t — there were multiple sets of registers. Hyperthreading is a form of virtualization, too. What typically happens is that people are running their cloud instances on machines whose underlying physical hardware is not fully utilized, and they get used to a certain level of performance they’re not really paying for. Alas, you can’t really know whether this is happening or not! But it surely is in many (most?) cases, which is why occasionally you get some resource that seems much slower than you’re accustomed to, and you think it’s “too slow.” Not so. Your other units are “too fast.” I have a theory that if you really knew the true capacity you were buying, you’d view the price-to-performance ratio much less favorably. But it’s almost impossible to know that, really; it doesn’t help that the cloud service providers are rather vague about how much power a certain instance size really gives you. (They aren’t being malicious; it’s just the way virtualization works.) Under-provisioning is almost forced on users because they have no alternative — you could plan for worst-case performance, and you’d be doing the right thing, but how will you ever know you’ve really hit rock bottom and the worst case is really no worse? How can you even benchmark and do proper capacity planning, if you don’t know what you’re benchmarking? This should really give you serious pause. You should be thinking “wait, I’m basing my capacity planning and provisioning on luck and the law of large numbers. What if my luck runs out and I get a Black Swan event?” The question is not “what if,” but “when.” I also think that the lack of transparency encourages people to use cloud computing services for the wrong reasons altogether. I could write about this, but I think Theo Schlossnagle said it pretty well already. Related posts:Drizzle stops the rainSessions of interest at the Percona Performance ConferenceWhy MySQL might not benefit from having a mother shipI moved this blog to pairLite with zero downtime, and it was easyHow good is the new High Performance MySQL going to be?";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:7:"pubDate";a:1:{i:0;a:5:{s:4:"data";s:31:"Wed, 02 Jun 2010 01:31:46 +0000";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:8:"category";a:6:{i:0;a:5:{s:4:"data";s:10:"Commentary";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:1;a:5:{s:4:"data";s:10:"PostgreSQL";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:2;a:5:{s:4:"data";s:3:"SQL";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:3;a:5:{s:4:"data";s:15:"Cloud Computing";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:4;a:5:{s:4:"data";s:17:"Theo Schlossnagle";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:5;a:5:{s:4:"data";s:14:"virtualization";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}s:40:"http://purl.org/rss/1.0/modules/content/";a:1:{s:7:"encoded";a:1:{i:0;a:5:{s:4:"data";s:6572:"

A common problem I see people running into when using a cloud computing service is the trap of under-provisioning. There’s a chain effect that leads to this result: 1) people don’t understand how virtualization works, and therefore 2) they don’t realize how much of a computing resource they’re really buying, so 3) they assume they are entitled to more than they really are, and 4) they under-provision. A few other causes and effects come into play here, too. For example, the choice to use the cloud is sometimes founded on economic assumptions that frequently turn out to be wrong. The cloud service looks more economically attractive than it really is, due to under-provisioning.

Let’s get back to this idea that people under-provision. How do I know that’s happening? I’ll use anecdotal evidence to illustrate. Here’s a real quote from a recent engagement about database (MySQL) performance problems:

Do you think it’s likely that the underlying hardware is simply worse than average? If you think this will be an ongoing problem, maybe we should try our luck with a new instance/storage cluster?

The fundamental assumption here is that some clusters are overloaded and are giving poor quality of service. We’re trained to think this way because we are familiar with services such as shared hosting, where other users on your particular server might really be abusive and claim resources that should be yours. But this isn’t how virtualization works in the common cloud platforms. In these platforms, you aren’t sharing resources with other users. You are guaranteed to get what you deserve! No kidding — this actually works.

If that’s true, then why does performance fluctuate so much? The answer lies in how resources are parceled out. Assume there are 10 units of computing resources, and you’re paying for one of them. You buy 1/10th of the machine’s power. But it just happens that you’re the only virtual instance running on that physical server. You fire up an intense job. How much power do you get? You paid for 1 unit, but you get 10, because no one else is using the other 9 units. This is the way most virtualization platforms work: they give you extra resources if they’re available and not being claimed by anyone else’s instance. This guarantees that you’ll never get less than you deserve, but it leaves open the possibility that you’ll get more than you deserve. (What would be the point of wasting that power, really?) Under-provisioning is the obverse of over-providing, which is what the virtualization platform does.

First-generation hyperthreading gave the same illusion of more resources than are really available, by the way. It made you think there were multiple processors, when in fact there weren’t — there were multiple sets of registers. Hyperthreading is a form of virtualization, too.

What typically happens is that people are running their cloud instances on machines whose underlying physical hardware is not fully utilized, and they get used to a certain level of performance they’re not really paying for. Alas, you can’t really know whether this is happening or not! But it surely is in many (most?) cases, which is why occasionally you get some resource that seems much slower than you’re accustomed to, and you think it’s “too slow.” Not so. Your other units are “too fast.”

I have a theory that if you really knew the true capacity you were buying, you’d view the price-to-performance ratio much less favorably. But it’s almost impossible to know that, really; it doesn’t help that the cloud service providers are rather vague about how much power a certain instance size really gives you. (They aren’t being malicious; it’s just the way virtualization works.) Under-provisioning is almost forced on users because they have no alternative — you could plan for worst-case performance, and you’d be doing the right thing, but how will you ever know you’ve really hit rock bottom and the worst case is really no worse? How can you even benchmark and do proper capacity planning, if you don’t know what you’re benchmarking? This should really give you serious pause. You should be thinking “wait, I’m basing my capacity planning and provisioning on luck and the law of large numbers. What if my luck runs out and I get a Black Swan event?” The question is not “what if,” but “when.”

I also think that the lack of transparency encourages people to use cloud computing services for the wrong reasons altogether. I could write about this, but I think Theo Schlossnagle said it pretty well already.

Related posts:

  1. Drizzle stops the rain
  2. Sessions of interest at the Percona Performance Conference
  3. Why MySQL might not benefit from having a mother ship
  4. I moved this blog to pairLite with zero downtime, and it was easy
  5. How good is the new High Performance MySQL going to be?


PlanetMySQL Voting: Vote UP / Vote DOWN";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}s:32:"http://purl.org/dc/elements/1.1/";a:1:{s:7:"creator";a:1:{i:0;a:5:{s:4:"data";s:22:"Baron Schwartz (xaprb)";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}}}i:48;a:6:{s:4:"data";s:48:" ";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";s:5:"child";a:3:{s:0:"";a:6:{s:5:"title";a:1:{i:0;a:5:{s:4:"data";s:47:"Check how old your MySQL books are before usage";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:4:"guid";a:1:{i:0;a:5:{s:4:"data";s:70:"tag:blogger.com,1999:blog-7603704315097619422.post-3774987396997346416";s:7:"attribs";a:1:{s:0:"";a:1:{s:11:"isPermaLink";s:5:"false";}}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:4:"link";a:1:{i:0;a:5:{s:4:"data";s:76:"http://geert.vanderkelen.org/2010/06/check-how-old-your-mysql-books-are.html";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:11:"description";a:1:{i:0;a:5:{s:4:"data";s:725:"This is a friendly reminder to check the publication date and discussed version you MySQL books before starting out hacking or even posting about limitations. Lots of old books are still going around. Maybe it's good to destroy them rather than giving them to students or newbies.Few days ago (28 May 2010), for example, we had a word-for-word copy of a book on a blog post (now removed) which was discussing MySQL Cluster limitations from years ago. Well, it was funny at first and we had a good laugh. But it's a bit worrisome. My colleague Matthew posted a rebuttal post.How would you recycle the old, technical books? It's not worth giving them to public libraries, it's maybe unhealthy to burn them? How would you do it?";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:7:"pubDate";a:1:{i:0;a:5:{s:4:"data";s:31:"Tue, 01 Jun 2010 18:24:00 +0000";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:8:"category";a:2:{i:0;a:5:{s:4:"data";s:7:"opinion";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:1;a:5:{s:4:"data";s:5:"mysql";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}s:40:"http://purl.org/rss/1.0/modules/content/";a:1:{s:7:"encoded";a:1:{i:0;a:5:{s:4:"data";s:1300:"

This is a friendly reminder to check the publication date and discussed version you MySQL books before starting out hacking or even posting about limitations. Lots of old books are still going around. Maybe it's good to destroy them rather than giving them to students or newbies.

Few days ago (28 May 2010), for example, we had a word-for-word copy of a book on a blog post (now removed) which was discussing MySQL Cluster limitations from years ago. Well, it was funny at first and we had a good laugh. But it's a bit worrisome. My colleague Matthew posted a rebuttal post.

How would you recycle the old, technical books? It's not worth giving them to public libraries, it's maybe unhealthy to burn them? How would you do it?


PlanetMySQL Voting: Vote UP / Vote DOWN";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}s:32:"http://purl.org/dc/elements/1.1/";a:1:{s:7:"creator";a:1:{i:0;a:5:{s:4:"data";s:17:"Geert Vanderkelen";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}}}i:49;a:6:{s:4:"data";s:63:" ";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";s:5:"child";a:3:{s:0:"";a:6:{s:5:"title";a:1:{i:0;a:5:{s:4:"data";s:34:"Eventually consistent Group Commit";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:4:"guid";a:1:{i:0;a:5:{s:4:"data";s:38:"http://ronaldbradford.com/blog/?p=2836";s:7:"attribs";a:1:{s:0:"";a:1:{s:11:"isPermaLink";s:5:"false";}}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:4:"link";a:1:{i:0;a:5:{s:4:"data";s:77:"http://ronaldbradford.com/blog/eventually-consistent-group-commit-2010-06-01/";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:11:"description";a:1:{i:0;a:5:{s:4:"data";s:1062:"Having just written an interview response about NoSQL concepts for a RDBMS audience it was poetic that an inconspicuous title “(4 of 3)” highlights that both a MySQL read scalable implementation via replication and a NoSQL solution can share a common lack of timely consistency of data. For the sake of Group Commit I hope my data is always consistent at some location at some point in time as soon as possible. In attempting to comment to Kristian Nielsen’s Fixing MySQL group commit (part 4 of 3) I was forced to watch an ad before I could even add a comment. Go jump Live Journal, it’s quicker to write my own blog post. And if anybody is still reading, I had just written the following. “There is clearly a place for NoSQL solutions. The two primary types of products are a key/value store and a schema-less solution. You need to learn the strengths, benefits and weaknesses of both. For a RDBMS resource the lack of transactions, the lack of joins and the concept of eventually consistent can take some time to accept.”";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:7:"pubDate";a:1:{i:0;a:5:{s:4:"data";s:31:"Tue, 01 Jun 2010 17:36:53 +0000";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}s:8:"category";a:5:{i:0;a:5:{s:4:"data";s:9:"Databases";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:1;a:5:{s:4:"data";s:5:"MySQL";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:2;a:5:{s:4:"data";s:12:"Professional";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:3;a:5:{s:4:"data";s:11:"consistency";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}i:4;a:5:{s:4:"data";s:5:"nosql";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}s:40:"http://purl.org/rss/1.0/modules/content/";a:1:{s:7:"encoded";a:1:{i:0;a:5:{s:4:"data";s:1374:"

Having just written an interview response about NoSQL concepts for a RDBMS audience it was poetic that an inconspicuous title “(4 of 3)” highlights that both a MySQL read scalable implementation via replication and a NoSQL solution can share a common lack of timely consistency of data. For the sake of Group Commit I hope my data is always consistent at some location at some point in time as soon as possible.

In attempting to comment to Kristian Nielsen’s Fixing MySQL group commit (part 4 of 3) I was forced to watch an ad before I could even add a comment. Go jump Live Journal, it’s quicker to write my own blog post.

And if anybody is still reading, I had just written the following.

“There is clearly a place for NoSQL solutions. The two primary types of products are a key/value store and a schema-less solution. You need to learn the strengths, benefits and weaknesses of both. For a RDBMS resource the lack of transactions, the lack of joins and the concept of eventually consistent can take some time to accept.”


PlanetMySQL Voting: Vote UP / Vote DOWN";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}s:32:"http://purl.org/dc/elements/1.1/";a:1:{s:7:"creator";a:1:{i:0;a:5:{s:4:"data";s:15:"Ronald Bradford";s:7:"attribs";a:0:{}s:8:"xml_base";s:0:"";s:17:"xml_base_explicit";b:0;s:8:"xml_lang";s:0:"";}}}}}}}}}}}}}}}}s:4:"type";i:128;s:7:"headers";a:7:{s:4:"date";s:29:"Wed, 09 Jun 2010 02:46:58 GMT";s:6:"server";s:22:"Apache/2.2.13 (Fedora)";s:13:"last-modified";s:29:"Wed, 09 Jun 2010 02:46:18 GMT";s:13:"accept-ranges";s:5:"bytes";s:14:"content-length";s:6:"322595";s:10:"connection";s:5:"close";s:12:"content-type";s:8:"text/xml";}s:5:"build";s:14:"20090627192103";}