Setting Null Values in Datomic
Datomic does not allow the storage of null values. We must reconcile which values were edited and which values were emptied. The emptied values must be a retracted to set to null.
Why Do Null Values Not Exist in Datomic?
It took a few moments to fully understand why it is that Datomic does not allow setting a value to null. If you do try, this is the exception you receive.
IllegalArgumentExceptionInfo :db.error/nil-value Nil is not a legal value datomic.error/arg (error.clj:55)
Looking at the development resources for Datomic and the section on Implications, it states:
Most Datomic writes are of tree nodes. These writes are compatible with eventually consistent storage, because the semantics of immutable values are beautifully simple: In an immutable system with no updates, there are only two possibilities: - a value is present - a value is not present yet
The important pieces to understand are the bullet points. A value must either exist or not; it cannot be null. Thinking in terms of datoms and entities, an entity either has a value for an attribute or the attribute does not exist for the entity. The entity cannot have an attribute with a value of null.
So what if we need to set a value to null?
How to Set Null Values
Let’s say that John fills out a form and includes his firstname and lastname. We would save this to Datomic as such.
(datomic/transact conn [{:db/id (datomic/tempid :db.part/user)
:user/firstname "John"
:user/lastname "Smith"}])
We can retrieve these values like so.
(datomic/q '[:find ?id ?firstname ?lastname
:in $
:where
[?id :user/firstname ?firstname]
[?id :user/lastname ?lastname]]
(datomic/db conn))
;; => #{[17592186045418 "John" "Smith"]}
(datomic/touch
(datomic/entity (datomic/db conn) 17592186045418))
;; => {:db/id 17592186045418, :user/firstname "John", :user/lastname "Smith"}
Now John wishes to remove his lastname from the system. Instead of setting the value of :user/lastname to null, we have to retract the fact of John’s lastname being “Smith”.
(datomic/transact conn [[:db/retract 17592186045418
:user/lastname "Smith"]])
And the result of retracting John’s lastname.
;; same query as above
(datomic/q '[:find ?id ?firstname ?lastname
:in $
:where
[?id :user/firstname ?firstname]
[?id :user/lastname ?lastname]]
(datomic/db conn))
;; => #{}
No entities with both the :user/firstname and :user/lastname attributes exist.
However.
(datomic/q '[:find ?id ?firstname
:in $
:where
[?id :user/firstname ?firstname]]
(datomic/db conn))
;; => #{[17592186045418 "John"]}
(datomic/touch
(datomic/entity (datomic/db conn) 17592186045418))
;; => {:db/id 17592186045418, :user/firstname "John"}
It’s the same entity but with the :user/lastname attribute retracted.
Retracting Values Programmatically
When John submits the form to remove his lastname, the system can’t hardcode the old value to build the retract statement. Here’s how you might go about finding the correct value and retracting when you need to. This code will not retract values which weren’t removed.
(defn edit-or-create-user-txn [params]
"Returns a transaction for creating a new entity and/or
updating attributes of an existing entity."
(let [user-entity (find-user-entity (:id params))
id (if user-entity
(:db/id user-entity)
(datomic/tempid :db.part/user -1))
edited (edited-fields id params)
retracted (retracted-fields user-entity params)]
(apply conj [] edited retracted)))
user=> (edit-or-create-user-txn {:user/firstname "John"
:user/middlename "Patrick"
:user/lastname "Smith"})
;; => [{:db/id 17592186045418
;; => :user/firstname "John"
;; => :user/middlename "Patrick"
;; => :user/lastname "Smith"}]
user=> (edit-or-create-user-txn {:id 17592186045418
:user/firstname "Johnny"
:user/middlename ""
:user/lastname ""})
;; => [{:db/id 17592186045418
;; => :user/firstname "Johnny"}
;; => [:db/retract 17592186045418 :user/middlename "Patrick"]
;; => [:db/retract 17592186045418 :user/lastname "Smith"]]
In order for the above function to work, we need edited-fields.
(defn edited-fields [id params]
"Returns a map of non-empty values to be used as a Datomic entity."
(into {:db/id id}
(filter second {:user/firstname (:firstname params)
:user/lastname (:lastname params)
...})))
And retracted-fields.
(defn- has-empty-value? [[k v]]
(empty? (str v)))
(defn- keys-for-empty-vals
"Get a list of keys from a map which have empty values."
[m]
(keys (filter has-empty-value? m)))
(defn- retract-statement
"Only build a retract statement for existing values."
[old-entity k]
(let [v (get old-entity k)]
(when v
[:db/retract (:db/id old-entity) k v])))
(defn retracted-fields
"Get all retractions from an existing entity."
[old-entity new-entity]
(filterv identity (map (partial retract-statement old-entity)
(keys-for-empty-vals new-entity))))
user=> (retracted-fields {
:user/firstname "John"
:user/middlename "Patrick"
:user/lastname "Smith"
}
{
:user/firstname "John"
:user/middlename ""
:user/lastname ""
})
;; => [[:db/retract 17592186045418 :user/middlename "Patrick"]
;; => [:db/retract 17592186045418 :user/lastname "Smith"]]
In a sort of round-a-about way, retracted-fields returns a vector of :db/retract vectors for datomic/transact to handle. It does so using the following method.
- Get a list of all keys in the
new-entitywhich have empty values. - Map across them to build a
[:db/retract eid key val]. - Filter out retraction vectors which were empty (i.e. no retraction needed).
Write Once, Use Everywhere
The nice thing about the retractions code is it’s been generalized enough that we’ve pulled it into it’s own namespace.
(ns com.example.retractions
...)
This makes it super simple for the next entity which requires nullable modifications.
(retractions/retracted-fields old-car new-car)