wr
hn
Top
New
Best
Ask
Show
Jobs
Back
d
digitaldragon
Joined 2 years ago
Karma
12
Submissions
2
Member since
Oct 17
View on Hacker News
Recent submissions
Comment · 1 year ago
The data is saved as a WARC file, which contains the entire HTTP request and response (compressed, of course). So it's much bigger than just a short -> long URL mapping.
Comment · 2 years ago
Unfortunately, Browsertrix relies on the Chrome Devtools Protocol, which strips transfer encoding (and possibly transforms the data in other ways). This results in Browsertrix writing noncompliant WARC files, because the spec requires that the original transfer encoding be preserved.