Branch: refs/heads/xrdec_repair
Home: https://github.com/xrootd/xrootd
Commit: 12551ee549cab25fedc4b7f627ce15b70a951850
https://github.com/xrootd/xrootd/commit/12551ee549cab25fedc4b7f627ce15b70a951850
Author: niklas <[log in to unmask]>
Date: 2022-08-25 (Thu, 25 Aug 2022)
Changed paths:
M src/XrdCl/CMakeLists.txt
M src/XrdCl/XrdClFileStateHandler.cc
M src/XrdCl/XrdClOperations.hh
M src/XrdCl/XrdClZipArchive.cc
M src/XrdCl/XrdClZipArchive.hh
M src/XrdCl/XrdClZipOperations.hh
M src/XrdEc/CMakeLists.txt
A src/XrdEc/XrdEcBlkPool.hh
M src/XrdEc/XrdEcObjCfg.hh
M src/XrdEc/XrdEcReader.cc
M src/XrdEc/XrdEcReader.hh
A src/XrdEc/XrdEcRepairTool.cc
A src/XrdEc/XrdEcRepairTool.hh
M src/XrdEc/XrdEcStrmWriter.cc
M src/XrdEc/XrdEcUtilities.hh
M src/XrdZip/XrdZipCDFH.hh
M src/XrdZip/XrdZipEOCD.hh
M src/XrdZip/XrdZipLFH.hh
M tests/XrdEcTests/MicroTest.cc
Log Message:
-----------
Compilable first draft of Repair Tool
Almost working except for the repair tool being destroyed due to a shared pointer
This version works quite well, but uses mtfile
Working version of the repair tool handles data and metadata corruption
rebased to master
Cleaned code
rebased to master
Last fixes
rebased to master
Restructured and commented repair tool functionality, the opening of archives can either be followed by a host replacement or on its own to detect and report on damaged archives to the user
Added callback for just checking a file's checksum, but has a bug with checksum comparison in reader
Fixed the archive opening and closing and the only instance that checks metadata is now the repair tool
Archive closing waits until everything is written using a pipeline final and condition locking
Fixed: We wait until all writes are finished before closing archives, WriteIntoFile is now a zip operation
Fix: Wait for all replacement archives to be opened
Added a test that corrupts two archives (or one archives in two spots), ran it successfully a couple hundred times
Fixes: Corrupt archives at correct offsets and replaced two usages of uncompressedSize by compressedSize
Fixed the actual root of the issue because corrupted offsets can lead to signflips in minus operations
Small concurrency fix and formatting
Concurrent version of repair
Fixes to make more than nbdata+nbparity hosts possible, changed unit test to use 2 hosts more
Cleaned some code
Further cleanup
Removed more unnecessary code and commented out lines
Added a test to confirm that repair fails when too much is corrupted
Added timeouts to the repair tool and a check in the opening process that checks whether one of the archives was blocked by the user. If it is the case, the tool exits immediately
Adding some timeouts seems to introduce a bug, might have to revert?
Trying to fix segfault for missing host
Fixed an error that the open pipeline would fail if not all readHealth are successful
Fixed the bugs, forgot a handler callback
Changed atomics to relaxed memory order and implemented a block pool to limit the memory consumption
Corrected a rebase mistake
Removed file repair functionality to split it into a second PR
Commit: 1b7c0af23bb8b3b23b9e2701ef11c41ffff3224e
https://github.com/xrootd/xrootd/commit/1b7c0af23bb8b3b23b9e2701ef11c41ffff3224e
Author: niklas <[log in to unmask]>
Date: 2022-08-25 (Thu, 25 Aug 2022)
Changed paths:
M src/XrdEc/XrdEcRepairTool.cc
Log Message:
-----------
Removed iostream include from XrdEcRepairTool.cc
Commit: a1e7f273daf8b0791ebc80c4c9dfb0de4713fb05
https://github.com/xrootd/xrootd/commit/a1e7f273daf8b0791ebc80c4c9dfb0de4713fb05
Author: niklas <[log in to unmask]>
Date: 2022-08-25 (Thu, 25 Aug 2022)
Changed paths:
M src/XrdCl/CMakeLists.txt
M src/XrdCl/XrdClOperations.hh
M src/XrdCl/XrdClZipArchive.cc
M src/XrdCl/XrdClZipOperations.hh
M src/XrdEc/XrdEcReader.cc
M src/XrdEc/XrdEcReader.hh
M src/XrdEc/XrdEcStrmWriter.cc
M src/XrdEc/XrdEcUtilities.hh
Log Message:
-----------
Corrected indentations and wildcard catches
Commit: 92bd3295156fc031d9b8831d660a1d9a024ab831
https://github.com/xrootd/xrootd/commit/92bd3295156fc031d9b8831d660a1d9a024ab831
Author: niklas <[log in to unmask]>
Date: 2022-08-25 (Thu, 25 Aug 2022)
Changed paths:
M src/XrdCl/XrdClOperations.hh
M src/XrdCl/XrdClZipArchive.hh
M src/XrdEc/XrdEcReader.cc
M src/XrdZip/XrdZipEOCD.hh
M src/XrdZip/XrdZipLFH.hh
M tests/XrdEcTests/MicroTest.cc
Log Message:
-----------
More indentation corrections
Commit: 839cfdb4ef6668b27b20b4c22a6ae3480610caa8
https://github.com/xrootd/xrootd/commit/839cfdb4ef6668b27b20b4c22a6ae3480610caa8
Author: niklas <[log in to unmask]>
Date: 2022-08-25 (Thu, 25 Aug 2022)
Changed paths:
M src/XrdEc/XrdEcObjCfg.hh
M src/XrdEc/XrdEcStrmWriter.cc
M tests/XrdEcTests/MicroTest.cc
Log Message:
-----------
Added random seed to objcfg to make server shuffling reproducable
Commit: 2f47c8e499cd8ace3dd6a1b7ad3eb1775acbd3dc
https://github.com/xrootd/xrootd/commit/2f47c8e499cd8ace3dd6a1b7ad3eb1775acbd3dc
Author: simonmichal <[log in to unmask]>
Date: 2022-08-30 (Tue, 30 Aug 2022)
Changed paths:
M src/XrdCl/XrdClFileStateHandler.cc
M src/XrdCl/XrdClZipArchive.cc
M src/XrdCl/XrdClZipArchive.hh
M src/XrdEc/CMakeLists.txt
A src/XrdEc/XrdEcBlkPool.hh
M src/XrdEc/XrdEcObjCfg.hh
M src/XrdEc/XrdEcReader.cc
M src/XrdEc/XrdEcReader.hh
A src/XrdEc/XrdEcRepairTool.cc
A src/XrdEc/XrdEcRepairTool.hh
M src/XrdEc/XrdEcStrmWriter.cc
M src/XrdEc/XrdEcUtilities.hh
M src/XrdZip/XrdZipCDFH.hh
M src/XrdZip/XrdZipEOCD.hh
M tests/XrdEcTests/MicroTest.cc
Log Message:
-----------
Merge pull request #1770 from Nikipedia/fileScannerPullReq
File Scanner Tool
Compare: https://github.com/xrootd/xrootd/compare/f937104050c6...2f47c8e499cd
########################################################################
Use REPLY-ALL to reply to list
To unsubscribe from the XROOTD-DEV list, click the following link:
https://listserv.slac.stanford.edu/cgi-bin/wa?SUBED1=XROOTD-DEV&A=1
|
