[jrk]

I am interested in directly adding content to my OmniFocus database by programmatically writing transaction files into my sync server data store. (Example use case: email-to-OmniFocus inbox, appearing immediately on an iOS device without needing a Mac to actually be online and fetching mail at the time.)

The format seems relatively simple (especially for simple item additions).

A few questions arise:

What is the exact naming scheme for the transaction files? It seems to be simply:

[date +%Y%m%d%H%M%s]=[base64encode(hash)].zip

What is the hash function, and what is hashed? Does this actually matter, or is it just there to create a safe unique ID for the file?

How is the timezone determined? (It doesn't seem to be UTC or local…)

Does file metadata matter (timestamp), or is everything just pulled from the name and contents?

Are the app-id, app-version, os-name, etc. parameters treated significantly by any of the clients at sync time (e.g. for schema migrations), or are they mostly informational? Would leaving these largely blank and specifying a custom app-id likely work?

Is the object ID significant in any way other than being unique? Would it be safe to generate another arbitrary unique base64 string when adding new items?

Thanks for entertaining some nerdy questions.

[Ken Case]

Originally Posted by jrk

What is the exact naming scheme for the transaction files? It seems to be simply:

[date +%Y%m%d%H%M%s]=[base64encode(hash)].zip

Close. It's actually [date]=[new-id]+[existing-id]+[existing-id].zip.

How is the timezone determined? (It doesn't seem to be UTC or local…)

The date should be expressed as yyyyMMddHHmmss in the UTC time zone.

What is the hash function, and what is hashed? Does this actually matter, or is it just there to create a safe unique ID for the file?

…

Is the object ID significant in any way other than being unique? Would it be safe to generate another arbitrary unique base64 string when adding new items?

The new id should be unique and limited to letters, numbers, dashes and underscores; if you avoid the lengths we use you shouldn't have to worry about name collisions, so you could probably just use a sequence of the form "MyTransactions-1" through "MyTransactions-9999" and beyond (since our ids are shorter than that).

The list of existing-ids (separated by plus signs) indicate where your new transaction should be attached to the current graph of transactions. (They should each be tails of the graph, with a common ancestor.) That lets OmniFocus know what you thought the database looked like when you made your change, making it easier to merge simultaneous changes from multiple devices. (Since you're primarily talking about adding new tasks, you probably don't have much need to worry about conflicts. Unless you're planning to add them to existing projects, in which case you could end up assigning them to a project or context which another client is in the process of deleting…)

On that note, you might also want to write client files to prevent other clients from deleting the transactions you were just about to use as your reference points. The parts to pay the most attention to are the date in the name and the array of tail identifiers, though if you go this route you'll also need to provide a client identifier, host id, last sync date, and registration date.

(If you don't go this route, you'll presumably need to double-check after writing to make sure the transactions you were expecting your new transaction to connect to are still there, and if not you'd clean up your aborted attempt and try again.)

Does file metadata matter (timestamp), or is everything just pulled from the name and contents?

Filesystem metadata doesn't matter, just the name and contents.

Are the app-id, app-version, os-name, etc. parameters treated significantly by any of the clients at sync time (e.g. for schema migrations), or are they mostly informational? Would leaving these largely blank and specifying a custom app-id likely work?

The app-version is used for schema migration, the rest is informational (unless we need to work around a platform-specific/app-specific bug at some point).

Thanks for entertaining some nerdy questions.

No problem at all; I hope you find this helpful!

[sprugman]

I'm interested in the same thing. What platform are you working on? Any interest in sharing effort?

[jrk]

I haven't gotten back to this, since I've been on the road since the reply, but many thanks, Ken. The fact that you guys give responses like this—let alone written by the CEO, himself—is part of why I love you so much.

Seriously.

I'll post code to GitHub and link back here when I get to take this anywhere.

[christophercomp]

Hey, jrk. I'm very interested in your idea. I find that having to send an email to my OmniFocus on my MacBook Pro from Windows or my Android device, then having to sync the MacBook Pro with the server before seeing on my iPad very cludgy.

I to want a way to send from my smartphone and have it show up instantly on the iPad after a sync.

I have some intermediate programming skills and have been poking around the OF transaction files for the last couple of days.

Let me know if you need any help at all and post when you get anything started.

[jrk]

I haven't gotten to this, since it hasn't been pressing for me now that I have an always-on Mac desktop in the mix, but I still hope to at some point. I hope to build it as a simple Sinatra app, hostable on Heroku, just relying on a pure Ruby DAV client library.