BenchmarkResultViews

Table of Contents Benchmark Result Views Examples View 1: Numbers of concept and join are fixed View 2: Numbers of concept and tuple are fixed View 3: Numbers of join and tuple are fixed View 4: Numbers of concept and constant are fixed View 5: Numbers of join and constant are fixed View 6: Numbers of tuple and constant are fixed

Benchmark Result Views

Six views are used to populate the measures used in the result analysis (see table below).

	Number of concept	Number of join	Number of tuple	Number of constant
View 1	×	×	•	•
View 2	×	•	×	•
View 3	•	×	×	•
View 4	×	•	•	•
View 5	•	×	•	×
View 6	•	•	×	×

Legend: • = A variable value, × = A fixed value

Each view corresponds to a specific database system (e.g., MySQL) and the experiment type (e.g., universal-semantic benchmark) by filtering based on the test_id. To construct the different views from the table, users can follow these guidelines:

1. For the fixed values, assign them as the filtering criteria in the WHERE clause.
2. For the variable values, assign them as the grouping criteria in the GROUP BY clause.

Users then have to change each fixed value to get the necessary coverage of the benchmark results. The specification of the test can be found in the Test table by selecting the appropriate test_id. For example, in one analysis scenario, we are focusing on a test_id: 20110318093451 (see image below).

In the highlighted row, the test configuration consists of a vector of joins of 2, 4 and 6; and a vector of concepts of 1, 3 and 5. If both parameters are the fixed values (i.e., View 1) then in order to have the full coverage of the benchmark results, users have to change the filtering criteria manually using the combination of both, e.g., [1, 2], [1, 4], [1, 6] and so on.

Examples

View 1: Numbers of concept and join are fixed

select expansion_id, database.tuples, database.constants, avg(execution_time) as average_time, stddev(execution_time) as std_deviation
from result left join database on database.db_id = result.db_id
where test_id = 20110318093451 and
      result.concepts = 1 and
      result.joins = 4
group by database.tuples, database.constants, expansion_id
order by database.tuples, database.constants

View 2: Numbers of concept and tuple are fixed

select expansion_id, result.joins, database.constants, avg(execution_time) as average_time, stddev(execution_time) as std_deviation
from result left join database on database.db_id = result.db_id
where test_id = 20110318093451 and
      result.concepts = 1 and
      database.tuples = 10000
group by result.joins, database.constants, expansion_id
order by result.joins, database.constants

View 3: Numbers of join and tuple are fixed

select expansion_id, result.concepts, database.constants, avg(execution_time) as average_time, stddev(execution_time) as std_deviation
from result left join database on database.db_id = result.db_id
where test_id = 20110318093451 and
      result.joins = 2 and
      database.tuples = 10000
group by result.concepts, database.constants, expansion_id
order by result.concepts, database.constants

View 4: Numbers of concept and constant are fixed

select expansion_id, result.joins, database.tuples, avg(execution_time) as average_time, stddev(execution_time) as std_deviation
from result left join database on database.db_id = result.db_id
where test_id = 20110318093451 and
      result.concepts = 1 and
      database.constants = 2500
group by result.joins, database.tuples, expansion_id
order by result.joins, database.tuples

View 5: Numbers of join and constant are fixed

select expansion_id, result.concepts, database.tuples, avg(execution_time) as average_time, stddev(execution_time) as std_deviation
from result left join database on database.db_id = result.db_id
where test_id = 20110318093451 and
      result.joins = 2 and
      database.constants = 2500
group by result.concepts, database.tuples, expansion_id
order by result.concepts, database.tuples

View 6: Numbers of tuple and constant are fixed

select expansion_id, result.concepts, result.joins, avg(execution_time) as average_time, stddev(execution_time) as std_deviation
from result left join database on database.db_id = result.db_id
where test_id = 20110318093451 and
      database.tuples = 10000 and
      database.constants = 2500
group by result.concepts, result.joins, expansion_id
order by result.concepts, result.joins