[SOLVED] Best way to open very large MBOX files?

General project discussion.
Use this as a last resort if your topic does not fit in any of the other boards but it still on-topic.
Forum rules
This General Discussion board is meant for topics that are still relevant to Pale Moon, web browsers, browser tech, UXP applications, and related, but don't have a more fitting board available.

Please stick to the relevance of this forum here, which focuses on everything around the Pale Moon project and its user community. "Random" subjects don't belong here, and should be posted in the Off-Topic board.
User avatar
Tharthan
Board Warrior
Board Warrior
Posts: 1442
Joined: 2019-05-20, 20:07
Location: New England

[SOLVED] Best way to open very large MBOX files?

Post by Tharthan » 2019-05-27, 23:44

I'm almost done with the stuff that I was trying to do computer-wise in that other thread, so I thought that I would just ask this question because I've wanted to know for some time now:

I have some Usenet archives that supposedly go back (a) decade(s) stored in the MBOX format.

When I tried to open one (I think that it was news.newusers.questions.mbox) that is about 528 Megabytes in file size, the program that I used (I'm not sure if I'm allowed to mention the program's name or not. It's a free MBOX reader, though) took a very long time (unsurprisingly) to open it, so I left it on overnight one night. When I got up, the program was hanging. I no longer have the error report, as I wiped general error report data some time ago, but I will note that when I cancelled the program's loading of the MBOX file partially through another time instead of letting it try to finish itself (which it just ends up not doing anyway, like I said, instead hanging), it does load a small number of the MBOX file that I can read, but it's all of the fairly recent stuff (earlier this decade), which I'm not that interested in.

So my question is: what is the best way to open very large MBOX files, particularly those as large as I just mentioned (528 Megabytes)?
Last edited by Tharthan on 2019-05-28, 07:48, edited 1 time in total.
"This is a war against individuality and intelligence. Only thing we can do is stand strong."adesh, 9 January 2020

"I used to think I was a grumpy old man, but I don't hold a candle compared to Tharthan."Cassette, 9 September 2020

User avatar
Moonchild
Pale Moon guru
Pale Moon guru
Posts: 38408
Joined: 2011-08-28, 17:27
Location: Motala, SE

Re: Best way to open very large MBOX files?

Post by Moonchild » 2019-05-28, 01:53

Whatever you choose, you likely want a 64-bit version to deal with something that large.
"There is no point in arguing with an idiot, because then you're both idiots." - Anonymous
"Seek wisdom, not knowledge. Knowledge is of the past; wisdom is of the future." -- Native American proverb
"Linux makes everything difficult." -- Lyceus Anubite

Michaell
Lunatic
Lunatic
Posts: 384
Joined: 2018-05-26, 18:13

Re: Best way to open very large MBOX files?

Post by Michaell » 2019-05-28, 03:04

MBOX files should be just text, except for some base64 encoded stuff like images, so any editor that handles large files should at least let you search for the text you want.

Notepad++ 64 bit opened the largest one of mine, 16MB, really fast. Noticeably faster than the Cool Utils Mbox Viewer, but of course it has nicer display.
Win10home(1709), PM33.9.0.1-portable as of Sep. 24, 2025

User avatar
Tharthan
Board Warrior
Board Warrior
Posts: 1442
Joined: 2019-05-20, 20:07
Location: New England

Re: Best way to open very large MBOX files?

Post by Tharthan » 2019-05-28, 07:46

Michaell wrote:
2019-05-28, 03:04
Notepad++ 64 bit opened the largest one of mine, 16MB, really fast. Noticeably faster than the Cool Utils Mbox Viewer, but of course it has nicer display.
Oh, wow! It actually opened even that 528 MB file in about 45 seconds or so. That's amazing, particularly because this archive (I have now discovered) goes back to January of 1997, which is pretty astonishing (although to be honest I was expecting earlier than that).

Thanks for the help, pal, and thanks to Moonchild as well for specifically recommending the use of a 64-bit version. Despite understanding on an intellectual level that there is a significant difference (I mean, compare even just in terms of functionality the Nintendo 64's CPU), I didn't manage (in my head) to link the dots, so it didn't occur to me that a 64-bit program would open something like this noticeably better than a 32-bit program.

Good to know going forward.
"This is a war against individuality and intelligence. Only thing we can do is stand strong."adesh, 9 January 2020

"I used to think I was a grumpy old man, but I don't hold a candle compared to Tharthan."Cassette, 9 September 2020

John connor

Re: [SOLVED] Best way to open very large MBOX files?

Post by John connor » 2019-06-02, 08:02

If you need to search for specific text in that file check out FileSeek.