[Beta] Syncing to new computer - attachments not showing

bwiernik

I'm setting up my library on a new computer. I'm alarmed to see that no items are showing the attachment circle as they are syncing down (I have attachments set to download as needed, but the behavior is the same if I switch to at sync time). Double clicking the item opens the URL/DOI, not downloads the attachment. Every previous time I've set up a new computer, the attachment icons have shown up immediately as the items synced down. Is this change in behavior expected?

bwiernik

Here is a few seconds of Debug output during the big initial sync:
D1617548649

dstillman

That's still a data sync — file syncing would happen after.

It should still download the item on-demand, though, so we'd want to see a Debug ID for that if it's not working.

bwiernik

Here is a Debug ID during a data sync where the I tried to click on an item to open the attachment, but nothing happens:
D837787141

From a user perspective, it seems weird that the attachment icons (not the files themselves) wouldn't show up in the data sync.

dstillman

I don't see a file-opening attempt there — it might have gone past the buffer. When you say nothing happens, do you mean that you get an error, or that nothing happens?

dstillman

From a user perspective, it seems weird that the attachment icons (not the files themselves) wouldn't show up in the data sync.

Actually, maybe I'm misunderstanding what you're saying. Do you just mean that you don't yet have child items (and that's why it's taking you to the URL)? Because that's true — 5.0 syncs parent items first, and then fills in the child items.

bwiernik

Yeah, that’s exactly it. Just found it a bit alarming. Thanks.

dstillman

Yeah, so that's largely a function of how the API works, and also from wanting to show ongoing progress during a sync, but we can consider not doing it that way.

The API allows getting all items, getting just top-level items, or getting the children of a given item. Since we want to show ongoing progress, we don't want to waste time fetching child items that we can't yet save, so we start by retrieving batches of top-level items. Once those are all downloaded, we fetch a list of all items, find those we haven't yet downloaded, and start requesting batches of those with the all-items request.

An alternative would be to download a batch of top-level items and then, before moving on to more top-level items, make requests for the child items of each of the top-level items that have them (which we can tell ahead of time). It would result in more requests overall, and so probably reduce the overall speed, but it would mean that child items would appear shortly after a top-level item appeared.

In either case, at-sync-time files wouldn't download until after all data syncing was done, but on-demand download would be available right away for the child attachments that had been saved.

The performance element probably isn't that big a deal — there are bigger ways we can optimize the sync anyway — so it's a question of what's more useful (and less alarming) to have sooner: complete items with attachments (useful if you want to open attachments) or all top-level items (useful if you want to search for something, cite something, or load a URL). You're probably right that the former would be more logical from a user perspective.

pjweiss

I would definitely vote for the former.

bwiernik

I think if at least the record of the child items could be retrieved so that the icon could be displayed (and the item double clicked to issue an interrupt file download request if needed), that would be preferred. With a large library, I worried that something had gone wrong with the syncing and that the attachments wouldn’t show up.

dstillman

OK, we'll see what we can do.