Lost message after Undo on a Move operation

[Navigator]

I moved a message from an IMAP Inbox to a local folder, then used the menu Undo command. The message is gone and I cannot find it anywhere now. Not in the Inbox, the local folder, IMAP Trash, or Local Folders Trash. Is this a known issue? Could a check be put in place to make sure the message is copied back before removing it, when trying to Undo a Move?

[Moonchild]

It depends very much on how the IMAP server is configured how (and if) this kind of undo/move back works. Keep in mind that IMAP is a server-based storage protocol, and local folders is a local storage location. There is no guarantee that the IMAP server will honour restoration of a message that is marked for deletion and if it doesn't, there's no guarantee that it will present a reasonable status to the client. The move operation may have been successful but the message actually not having been restored on the server (and the next update actually removing it from the cache).

This is one of the main reasons I tend to favour POP3, myself; move operations will always work as-expected because everything is client-side. And you would still have the option to keep messages mirrored server side (for a configurable length of time).

[athenian200]

I would probably advise against using "move" instead of "copy" for this reason. In general, I am the type of person who thinks it's safer to make a copy and then delete the original after verifying everything, than to do a move operation and have the delete/copy happen in one step without any guarantee of being able to go back to a valid state if there's an issue.

Note that this aspect of Epyrus was, yet again, designed by someone who isn't me, and thus this way of looking at things isn't enforced. And there are likely people who are comfortable enough with this quirk that if I tried to fix it by removing the "move" command in favor of copy and delete as independent operations, someone would likely complain.

[Bilbo47]

if I tried to fix it by removing the "move" command ... someone would likely complain.

Sadly yes but the trick would be to "eliminate the possibility or likelihood of data loss" by greying out the "move" option (and No-Opping the keyboard command) only when the source and destination locations are different as far as local versus remote. If this bug were confirmed then that's a change I can get behind.

Either way, I agree this sort of advanced operation is unfortunately to be avoided, if for no other reason than it was never a reliable feature-set in classical Thunderbird, possibly because of the reasons MC mentioned above (servers are inconsistent with how they handle the commands / implement the functionality).

[Moonchild]

Sorry but I would be very unhappy if move from IMAP to local storage was removed/disabled - having to copy then reselect and delete is IMO at greater risk of data loss because it involves more steps with big chance of mistakes in the selection steps.

What I think would be a good thing to do would be to prevent operations that try to move items from an external location (other IMAP or local folder) to an IMAP server folder since that normally doesn't work (it's not webdav or what not...) but this shouldn't be blocked for moving within the same IMAP server (as that's handled properly). I also think this might already be present, just not restricted under "undo"?

[Navigator]

I removed the resolved/accepted mark as it seems this is worth discussing after all. I am able to both Copy and Move messages from a Local Folder into this IMAP Inbox successfully so that is not a blocked operation. However using Undo after Moving a message from that IMAP Inbox to a local folder causes the message to disappear from the local folder without reappearing in the Inbox so there is something amiss. This is a repeatable failure as I tried it with several test messages.

Can we find out what is different between a manual Move from Local to IMAP, and an Undo?

[athenian200]

I removed the resolved/accepted mark as it seems this is worth discussing after all. I am able to both Copy and Move messages from a Local Folder into this IMAP Inbox successfully so that is not a blocked operation. However using Undo after Moving a message from that IMAP Inbox to a local folder causes the message to disappear from the local folder without reappearing in the Inbox so there is something amiss. This is a repeatable failure as I tried it with several test messages.

Can we find out what is different between a manual Move from Local to IMAP, and an Undo?

The thing is, this actually isn't consistent for me. I actually have copied things from Local Folders to an IMAP server and had it work... in fact, I used that to send myself notifications at one point, by collecting RSS feeds using Epyrus and then moving the local mail into my inbox. Those RSS feeds are stored as Local Folders... so it does work for me, though they do rarely get marked as spam.

So are you saying that the "undo" itself is the problem, and that this otherwise works?

I'm honestly not sure how much of this I can fix... I just don't have the level of familiarity with the Epyrus codebase I would need to fix most of the problems with it, and honestly the more I hear about the kind of nagging little issues people have, the more I think I've gotten myself in over my head. Still, I do want to familiarize myself with the problem in case one day I do figure out how to solve it.

The only easy fix I can think of is to just not allow undo operations that involve remote folders... I feel like an "undo" command only makes sense if you're working locally/offline, sort of like discarding commits you haven't pushed yet.

[Navigator]

So are you saying that the "undo" itself is the problem, and that this otherwise works?

I'm honestly not sure how much of this I can fix... I just don't have the level of familiarity with the Epyrus codebase I would need to fix most of the problems with it, and honestly the more I hear about the kind of nagging little issues people have, the more I think I've gotten myself in over my head. Still, I do want to familiarize myself with the problem in case one day I do figure out how to solve it.

Yes I believe Undo itself is the problem, and that yes it works otherwise, at least from testing this with several messages. I am not the one trying to crawl through the codebase but you have my sympathy. I don't want these "nagging little issues" to cause you to throw in the towel and walk away from the project. I simply don't know how much work any given issue would require. I can accept that some things are not worth the time it would take you.

Do you know if it would be reasonably easy to disable the Undo command on any Move operation that crosses the Server/Local boundary?

[athenian200]

Yes I believe Undo itself is the problem, and that yes it works otherwise, at least from testing this with several messages. I am not the one trying to crawl through through codebase but you have my sympathy. I don't want these "nagging little issues" to cause you to throw in the towel and walk away from the project. I simply don't know how much work any given issue would require. I can accept that some things are not worth the time it would take you.

Do you know if it would be reasonably easy to disable the Undo command on any Move operation that crosses the Server/Local boundary?

Don't worry, it won't cause me to throw in the towel. Really, the big issue I have is that a lot of my e-mail accounts that I use every day simply don't work with Epyrus.

That sounds like it should be easy enough to do in theory, but I would in fact have to find where in the codebase this is and whether it has checks for that scenario already... move operations crossing the server/local boundary. If there are such checks, it's possible they don't get repeated on undo, or a local folder being involved causes the code not to be run when it should be... like maybe there's backwards logic somewhere checking if a local folder is involved, and then not doing certain checks, when it should be checking if a remote folder is involved, and then performing the checks.

Do you know if the "undo" command works fine within an IMAP folder, as long as nothing moves between a local folder and an IMAP folder?

[Navigator]

Do you know if the "undo" command works fine within an IMAP folder, as long as nothing moves between a local folder and an IMAP folder?

No, but I'll test that now and edit this post with the result.

Okay, I tried it with two messages moved within the IMAP tree on a single account, and the Undo was successful on both.

Then I tried moving a message from the IMAP Inbox of one account to the IMAP folder tree of a second account. Here the Undo command seemed to have no effect, leaving the message in the destination folder on the second account, neither deleting it nor making it reappear in the Inbox of the first account. I then did a manual drag-and-drop Move to put the message back in the origin account Inbox and it worked as expected.

In summary I think we need to disable Undo on Move when that Move is not within a single folder tree, be it a server folder tree or the local folder tree. Unless of course you are able to figure out why the automatic move-back operation fails when the manual one works as expected.

[athenian200]

In summary I think we need to disable Undo on Move when that Move is not within a single folder tree, be it a server folder tree or the local folder tree. Unless of course you are able to figure out why the automatic move-back operation fails when the manual one works as expected.

That doesn't sound like a bad idea. It sounds like it could be simple depending on how the code is setup... just making move irreversible and maybe display a warning depending on context, is probably a lot easier than having to disable move altogether, and will disrupt workflows less for sure.

I'm going to be honest, I didn't even know Epyrus had an undo command... which is likely why I haven't kicked the tires on that functionality more.

[Navigator]

I am bumping this for attention. I think this warrants a fairly high priority, as though avoidable with foreknowledge having messages disappearing is a pretty serious bug/problem IMHO.

[athenian200]

I am bumping this for attention. I think this warrants a fairly high priority, as though avoidable with foreknowledge having messages disappearing is a pretty serious bug/problem IMHO.

Well, I did some research on Bugzilla, and found this...

https://bugzilla.mozilla.org/show_bug.cgi?id=1847392

It looks like modern Thunderbird still has the issue you pointed out in Epyrus. So it doesn't appear Mozilla has a solution for this, and we really are on our own with solving this. After a little bit of further research, I found a post on another related bug that suggests this has been an issue since Thunderbird 12 and is still not fixed in modern Thunderbird. It also can't be reproduced consistently and always gets closed as "Works for me," but people keep reporting it so there are definitely circumstances that trigger it.