Matthew Boston

Datomic performance with clause ordering

August 13, 2014

Key takeaway: Order Datomic :where clauses so the one that matches the fewest datoms comes first, then measure with your own data.

Put the most restrictive clause first in a Datomic query. The docs say so, and when I timed it, the gap grew as the data grew.

Say there are 100 cities, each with a daily temperature reading for the past year. That’s 36,500 entries. To find Denver’s temperature on July 31, you could write the query two ways:

[:find ?temp
 :in ?city ?date
 :where
 [?temp :temperature/city ?city]
 [?temp :temperature/date ?date]]
[:find ?temp
 :in ?city ?date
 :where
 [?temp :temperature/date ?date]
 [?temp :temperature/city ?city]]

The first narrows 36,500 entries to 365 for Denver, then to 1. The second narrows them to 100 for July 31, then to 1.

I ran each query 100 times. With one year of data, the city-first version averaged 8.91 msecs and the date-first version 7.47 msecs. That’s about a 16% difference.

With two years, 73,000 entries, city-first leaves 730 candidates after the first clause. Date-first still leaves 100. City-first averaged 18.87 msecs and date-first 12.53 msecs, about two thirds of the time. Clause order mattered more as the data grew.

It can go the other way, though. With only a week of data, city-first narrows 700 entries to 7 and date-first narrows them to 100, because there are more cities than dates. The difference there is fractions of a millisecond, but it shows that “most restrictive” depends on your data.