The Omni Group
These forums are now read-only. Please visit our new forums to participate in discussion. A new account will be required to post in the new forums. For more info on the switch, see this post. Thank you!

Go Back   The Omni Group Forums > OmniOutliner > OmniOutliner for iPad
FAQ Members List Calendar Search Today's Posts Mark Forums Read

 
Automatic document synchronization? Dropbox integration? Thread Tools Search this Thread Display Modes
We tried adding support for DropBox syncing in OmniOutliner, but that resulted in corrupt documents. See my post in this thread:

The challenge of syncing OmniOutliner
 
Thank you for that, but it feels like you missed my thesis: "Use file locks if you have to, provide import and export buttons, hack something together, but... SUPPORT DROPBOX"

You are trying to refactor your system to allow for atomic operations in a synced environments. I am no stranger to the issues that result from this. You are not wrong, it is very tricky to do perfectly. However they are also only likely to occur when true simultaneous access is occurring, and that is a rare situation.

Nor is it unique to Dropbox. Software has had to contend with these issues for decades. And yet we deal with it.

So by all means, continue to strive for perfect, but in the mean time why don't you implement good enough. Add import/export support for Dropbox, do some kind of file locking such that only one app can actually edit at a time, save to a shadow file and then play the rename game, etc (see Word's behavior in this situation for a perfectly acceptable solution).

If you toss us a bone then we can actually use your software. If you don't, and keep pursuing perfect at the expense of good enough, then you lose revenue, you lose customers, and you create active badwill.

There is always a solution. It may not be the one you want, but there is always a solution. That is what we care about.
 
Quote:
Originally Posted by JockM View Post
So by all means, continue to strive for perfect, but in the mean time why don't you implement good enough. Add import/export support for Dropbox, do some kind of file locking such that only one app can actually edit at a time, save to a shadow file and then play the rename game, etc (see Word's behavior in this situation for a perfectly acceptable solution).
We've actually tried many of these approaches with DropBox, including renaming (like we do with WebDAV), file locks, and so on. The problem is that DropBox not only doesn't support atomic transactions, it doesn't guarantee any order or synchronicity to its transactions. So you might think the document is not locked, but it is. (Or you might think it is when it isn't.) Or you tell it to rename, and your rename happens after the transfer instead of before. The synchronized directory looks fine for a little while—and then suddenly some extra files resurrect themselves inside it. We would love to automatically synchronize using DropBox, but it just doesn't seem to be designed to synchronize directories based on a specific point in time.

That said, DropBox does handle zip files perfectly well, so in our most recent update we added support for opening zip files (letting you send documents from the DropBox app to OmniOutliner) and in our next update we're adding support for sending zip archives of outlines to other apps (letting you send documents from OmniOutliner back to the DropBox app). Not an ideal workflow, but it's certainly an improvement.
 
This isn't really the place for an in depth technical conversation. But I will share this little tidbit: if you always use the dropbox REST API to write and test the existence of a lock file, then it will work consistantly
 
Quote:
Originally Posted by JockM View Post
This isn't really the place for an in depth technical conversation. But I will share this little tidbit: if you always use the dropbox REST API to write and test the existence of a lock file, then it will work consistantly
Huh! My understanding is that DropBox stores its data on Amazon S3, and the Amazon S3 data consistency model explicitly does not support read-after-write consistency (such as what we'd need to be able to rely on lock file tests).

Are you saying that DropBox's REST API makes consistency guarantees not made by the underlying Amazon S3 data store?
 
This is all based on empirical evidence (a boatload of scenario testing I did a year or so back), and tidbits I have put together from the APIs and the dropbox forums.

But Dropbox keeps a big old cache of metadata. While it get into the write-behind cache to S3, and synced to computers in whatever order happens behind the scenes, the metadata cache seems to get updated promptly.

So it isn't possible to inspect the contents of a file right away, in a consistant and guaranteed manor, creating and testing the existence of a locking-file shouldn't be an issue.

* * *

But I also don't want to get too bogged down into the minutiae of Dropbox (though if what I said helped, then I am only too happy). I have great respect for everyone at Omni, especially as engineers.

But as a user I don't really care about the challenges, I care that there is at least some low-friction way to get my data back to my desktop (as well as other iOS devices). And a "perfect" solution isn't required to do that.

A simple import/export feature would do that. Alternatively, since you mentioned writing zip files, then add the ability to open them as a first class file type as well as an OO Bundle (OmniOutliner will explode and recompress as needed behind the scenes).

This latter scenario doesn't require any Dropbox specific trickery, is relatively simple to implement, solves any issues with atomacy, and is unlikely to posses any unwanted side effects. It also opens the door ever so slightly for Windows applications to be able to interact with OO "files". Its a win/win all around.

Few (software) problems are actually impossible. There is always a solution of some sort. More often than not it involves changing the way you look at the problem.

Last edited by JockM; 2012-04-16 at 10:17 PM..
 
Quote:
Originally Posted by Ken Case View Post
We've actually tried many of these approaches with DropBox, including renaming (like we do with WebDAV), file locks, and so on. The problem is that DropBox not only doesn't support atomic transactions, it doesn't guarantee any order or synchronicity to its transactions. So you might think the document is not locked, but it is. (Or you might think it is when it isn't.) Or you tell it to rename, and your rename happens after the transfer instead of before. The synchronized directory looks fine for a little while—and then suddenly some extra files resurrect themselves inside it. We would love to automatically synchronize using DropBox, but it just doesn't seem to be designed to synchronize directories based on a specific point in time.

That said, DropBox does handle zip files perfectly well, so in our most recent update we added support for opening zip files (letting you send documents from the DropBox app to OmniOutliner) and in our next update we're adding support for sending zip archives of outlines to other apps (letting you send documents from OmniOutliner back to the DropBox app). Not an ideal workflow, but it's certainly an improvement.
On Omni's FAQ page it clearly states that Omni has no plans to support Dropbox so any and all arguments to the contrary are moot, or at least muted. Still, a thought: I don't care one iota about the details of file locks and what not.

Dropbox is supported by every app I use except Omni apps, it is a solution for me that is available everywhere and constantly. I have installed WebDAV links/shortcuts etc for one company's apps only: Omni's. It doesn't matter how good/bad Dropbox/WebDAV is for synching, I can handle my own synching without a problem and with Dropbox.

I'm probably in the minority here but it seems to me that the lack of simple Dropbox support is hurting the common user by forcing her to work Omni's way rather than supportive of that user's existing way of working. We know how Dropbox works, it's limitations, and we still want it. Period.
 
I know zip about software engineering, but while reading this thread I keep coming back to the same questions about Dropbox support.

How does CarbonFin do it? Is OO that different?
 
Quote:
Originally Posted by ryangoldman View Post
How does CarbonFin do it? Is OO that different?
Fundamentally it's that CarbonFin doesn't do attachments.

If you have an OO document with several attachments, it's not just one file, it's several. If there's a write error "at some point" while saving the document to the cloud, it may well be broken, and DropBox doesn't allow reverting several files after a single write error on one of them.

A proper sync solution will have to deal with this gracefully.
 
Quote:
Originally Posted by HappyCatMachine View Post
I'm probably in the minority here but it seems to me that the lack of simple Dropbox support is hurting the common user by forcing her to work Omni's way rather than supportive of that user's existing way of working. We know how Dropbox works, it's limitations, and we still want it. Period.
Well, we all know our limitations but we still want to be rich and beautiful too. It can be tough when one has to confront reality sometimes....

J

Last edited by Brian; 2012-05-14 at 03:34 PM.. Reason: reduced the amount of quoted material for increased readability
 
 


Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes


Similar Threads
Thread Thread Starter Forum Replies Last Post
DropBox support? [A: Document sync is coming, but not based on Dropbox or iCloud.] sw001 OmniGraffle for iPad 40 2013-01-13 05:00 PM
OmniFocus for iPad and integration with Dropbox/iCal barsiwek OmniFocus for iPad 4 2011-10-18 12:09 PM
Sync with new Dropbox API [DropBox is great, but not well-suited to OF sync.] kint OmniFocus Syncing 24 2010-08-05 01:24 PM
Automatic Layout with Automatic Re-alignment michael.prescott OmniGraffle General 0 2010-03-10 06:27 AM


All times are GMT -8. The time now is 11:35 PM.


Powered by vBulletin® Version 3.8.7
Copyright ©2000 - 2024, vBulletin Solutions, Inc.