You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
With the introduction of State Archival in p23, we are changing the History Archive format. This will require the corresponding changes in stellar-archivist. Given there's been lots of issues with the tool and it's an outdated go library, this may be a good opportunity to just rewrite it in something like rust and have the core team maintain it more actively going forward.
The text was updated successfully, but these errors were encountered:
We got some feedback from Blockdaemon around specific limitations of the current implementation:
Issues
In some modes, the tool treats corrupted files (e.g. zero length) as ok but the archive is bad (and SDF correctly flags the archive as bad)
replacing corrupted files with the archive tool is not intuitive, it has to be done it manually (blockdaemon has a workaround which involves doing explicit repair which will force overwrite existing files)
any upstream HTTP error (503, 429) results in the archive tool immediately aborting its run (blockdaemon observed that the errors and non errors are all on the same output, so you have to dig through a ton of logs to figure anything out and determine both where to pick up where you left off, and which files are missing/corrupted)
Both scan and repair should know the difference between zero length files, corrupted files, and correct files.
Ease of use: simplify the tool (which would make the documentation simpler); e.g., a scan should suggest the commands that need to be run to repair an archive.
Improve the ease of use of the tool by documenting the process in a simple way to detect and fix a file or ledger in the archive E.g. it would be good to know that the Archive is 500Tb, so operators should not keep it on their local disks. Explicitly document the workflow involved and add recommendations.
With the introduction of State Archival in p23, we are changing the History Archive format. This will require the corresponding changes in stellar-archivist. Given there's been lots of issues with the tool and it's an outdated go library, this may be a good opportunity to just rewrite it in something like rust and have the core team maintain it more actively going forward.
The text was updated successfully, but these errors were encountered: