{"id":2284,"date":"2013-10-13T17:45:27","date_gmt":"2013-10-13T21:45:27","guid":{"rendered":"http:\/\/hstore.cs.brown.edu\/?page_id=2284"},"modified":"2013-10-13T18:40:20","modified_gmt":"2013-10-13T22:40:20","slug":"automatic-database-designer","status":"publish","type":"page","link":"https:\/\/hstore.cs.brown.edu\/documentation\/deployment\/automatic-database-designer\/","title":{"rendered":"Automatic Database Designer"},"content":{"rendered":"<p id=\"top\" \/><div class=\"prevpage\"><B>\u00ab<\/B> <a href=\"https:\/\/hstore.cs.brown.edu\/documentation\/deployment\/deferred-execution\/\" title=\"Deferred Query Execution\">Deferred Query Execution<\/a><\/div> <div class=\"nextpage\"><a href=\"https:\/\/hstore.cs.brown.edu\/documentation\/deployment\/jvm-snapshots\/\" title=\"OLAP JVM Snapshots\">OLAP JVM Snapshots<\/a> <B>\u00bb<\/B><\/div><div style=\"clear:both;\"><\/div><\/p>\n<p>The following are instructions on how to use H-Store&#8217;s database design tool, called <em>Horticulture<\/em>. This work is was part of the <a href=\"projects\/partitioning\/\">automatic database partitioning project<\/a>.<\/p>\n<h2>Source Code Overview<\/h2>\n<p>All of the source code for Horticulture is available in the <a href=\"https:\/\/github.com\/apavlo\/h-store\/tree\/master\/src\/frontend\/edu\/brown\/designer\">main source code repository<\/a>.<\/p>\n<p>The design tool consists of two components: (1) the <b>search algorithm<\/b> and (2) the <b>cost model<\/b>. The search algorithm is the process that Horticulture uses to find the best design for an arbitrary application. The main large-neighborhood search algorithm described in the SIGMOD paper is <a href=\"https:\/\/github.com\/apavlo\/h-store\/blob\/master\/src\/frontend\/edu\/brown\/designer\/partitioners\/LNSPartitioner.java\">here<\/a>. This algorithm executes multiple rounds of a <a href=\"https:\/\/github.com\/apavlo\/h-store\/blob\/master\/src\/frontend\/edu\/brown\/designer\/partitioners\/BranchAndBoundPartitioner.java\">branch-and-bound search algorithm<\/a>.<\/p>\n<p>The search is guided by a cost model (i.e., objective function). The <a href=\"https:\/\/github.com\/apavlo\/h-store\/blob\/master\/src\/frontend\/edu\/brown\/costmodel\/SingleSitedCostModel.java\">default cost model<\/a> for Horticulture computes the ratio of the number of single-partition transactions to distributed transactions in the sample workload.<\/p>\n<h2>Workload Traces<\/h2>\n<p>The sample workloads that used in the experiments are available in H-Store&#8217;s <a href=\"https:\/\/database.cs.brown.edu\/svn\/hstore-files\/workloads\/\">supplemental file repository<\/a>. See these <a href=\"documentation\/deployment\/trace-logs\/\">instructions<\/a> on how to create these workload files for other benchmark implementations in H-Store.<\/p>\n<h2>Setup<\/h2>\n<p>Once you have the workloads and your project jar, you can create the &#8220;mappings&#8221; and &#8220;stats&#8221; files automatically using this command:<\/p>\n<pre>ant <tt class=\"ant-target\">designer-prepare<\/tt> -Dproject=$BENCHMARK<\/pre>\n<p>And then you can invoke the designer code:<\/p>\n<pre>ant <tt class=\"ant-target\">designer-benchmark<\/tt> -Dproject=$BENCHMARK<\/pre>\n<h2>Execution<\/h2>\n<pre>\r\nant <tt class=\"ant-target\">designer-benchmark<\/tt> -Dproject=$BENCHMARK \\\r\n   -Dworkload=files\/workloads\/${BENCHMARK}.trace.gz \\\r\n   -Dstats=files\/stats\/${BENCHMARK}.stats.gz \\\r\n   -Dlimit=5000 \\\r\n   -Dhints=files\/designhints\/${BENCHMARK}.hints \\\r\n   -Dpartitioner=edu.brown.designer.partitioners.BranchAndBoundPartitioner \\\r\n   -Dcostmodel=edu.brown.costmodel.SingleSitedCostModel\r\n<\/pre>\n<p>Options:<\/p>\n<ul>\n<li> The <tt class=\"ant-option\">limit<\/tt> option limits the number of txn traces to load from the sample workload.\n<li> The <tt class=\"ant-option\">hints<\/tt> file is a JSON file that contains additional config options. I made it a separate file instead of using commandline arguments to make it easier for me to prototype things.\n<li> The <tt class=\"ant-option\">partitioner<\/tt> option sets the class file to use for the partitioning algorithm. As you can see in <a href=\"https:\/\/github.com\/apavlo\/h-store\/tree\/master\/src\/frontend\/edu\/brown\/designer\/partitioners\">src\/frontend\/edu\/brown\/designer\/partitioners<\/a>, there are other algorithms:\n<ul>\n<li> <a href=\"https:\/\/github.com\/apavlo\/h-store\/tree\/master\/src\/frontend\/edu\/brown\/designer\/partitioners\/MostPopularPartitioner.java\">MostPopularPartitioner<\/a> &#8211; Choose the partitioning column for each table as the one that is most accessed by queries in the sample workload.\n<li> <a href=\"https:\/\/github.com\/apavlo\/h-store\/tree\/master\/src\/frontend\/edu\/brown\/designer\/partitioners\/PrimaryKeyPartitioner.java\">PrimaryKeyPartitioner<\/a> &#8211; Choose each table&#8217;s primary key as the partitioning column.\n<li> <a href=\"https:\/\/github.com\/apavlo\/h-store\/tree\/master\/src\/frontend\/edu\/brown\/designer\/partitioners\/RandomPartitioner.java\">RandomPartitioner<\/a> &#8211; Choose a random column in each table as its partitioning column.\n<li> <a href=\"https:\/\/github.com\/apavlo\/h-store\/tree\/master\/src\/frontend\/edu\/brown\/designer\/partitioners\/BranchAndBoundPartitioner.java\">BranchAndBoundPartitioner<\/a> &#8211; Run a branch-and-bound search to find the optimal partitioning scheme for the database. This is an exhaustive search.\n<li> <a href=\"https:\/\/github.com\/apavlo\/h-store\/tree\/master\/src\/frontend\/edu\/brown\/designer\/partitioners\/LNSPartitioner.java\">LNSPartitioner<\/a> &#8211; Run multiple rounds of the <tt>BranchAndBoundPartitioner<\/tt> to find the optimal partitioning scheme for a database.\n   <\/ul>\n<li> The <tt class=\"ant-option\">costmodel<\/tt> option sets the class file for objective function to use in the designer. <a href=\"https:\/\/github.com\/apavlo\/h-store\/blob\/master\/src\/frontend\/edu\/brown\/costmodel\/SingleSitedCostModel.java\">SingleSitedCostModel<\/a> is the basic one that will count the ratio of the number of single-partition txns with dtxns. This is probably the one you&#8217;ll want to use. There is a more complex one called <a href=\"https:\/\/github.com\/apavlo\/h-store\/blob\/master\/src\/frontend\/edu\/brown\/costmodel\/TimeIntervalCostModel.java\">TimeIntervalCostModel<\/a> that can handle discrete intervals when computing skew.\n<\/ul>\n<p><div class=\"prevpage\"><B>\u00ab<\/B> <a href=\"https:\/\/hstore.cs.brown.edu\/documentation\/deployment\/deferred-execution\/\" title=\"Deferred Query Execution\">Deferred Query Execution<\/a><\/div> <div class=\"nextpage\"><a href=\"https:\/\/hstore.cs.brown.edu\/documentation\/deployment\/jvm-snapshots\/\" title=\"OLAP JVM Snapshots\">OLAP JVM Snapshots<\/a> <B>\u00bb<\/B><\/div><div style=\"clear:both;\"><\/div><\/p>\n","protected":false},"excerpt":{"rendered":"<p>The following are instructions on how to use H-Store&#8217;s database design tool, called Horticulture. This work is was part of the automatic database partitioning project. Source Code Overview All of the source code for Horticulture is available in the main source code repository. The design tool consists of two components: (1) the search algorithm and [&hellip;]<\/p>\n","protected":false},"author":2,"featured_media":0,"parent":626,"menu_order":999,"comment_status":"closed","ping_status":"open","template":"","meta":[],"_links":{"self":[{"href":"https:\/\/hstore.cs.brown.edu\/wp-json\/wp\/v2\/pages\/2284"}],"collection":[{"href":"https:\/\/hstore.cs.brown.edu\/wp-json\/wp\/v2\/pages"}],"about":[{"href":"https:\/\/hstore.cs.brown.edu\/wp-json\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"https:\/\/hstore.cs.brown.edu\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/hstore.cs.brown.edu\/wp-json\/wp\/v2\/comments?post=2284"}],"version-history":[{"count":15,"href":"https:\/\/hstore.cs.brown.edu\/wp-json\/wp\/v2\/pages\/2284\/revisions"}],"predecessor-version":[{"id":2292,"href":"https:\/\/hstore.cs.brown.edu\/wp-json\/wp\/v2\/pages\/2284\/revisions\/2292"}],"up":[{"embeddable":true,"href":"https:\/\/hstore.cs.brown.edu\/wp-json\/wp\/v2\/pages\/626"}],"wp:attachment":[{"href":"https:\/\/hstore.cs.brown.edu\/wp-json\/wp\/v2\/media?parent=2284"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}