Skip to content

HBASE-29991 [HBCK2] offlineReferenceFileRepair need execute 2 times t… #161

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
chaijunjie0101 wants to merge 1 commit into
base: master
Choose a base branch
from
+21 −13
Open

HBASE-29991 [HBCK2] offlineReferenceFileRepair need execute 2 times t… #161

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
34 changes: 21 additions & 13 deletions hbase-hbck2/src/main/java/org/apache/hbase/hbck1/HBaseFsck.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1208,11 +1208,11 @@ private void offlineReferenceFileRepair() throws IOException, InterruptedExcepti
}

/**
* Runs through the HBase rootdir and creates a reverse lookup map for table StoreFile names to
* the full Path. <br>
* Runs through the HBase rootdir and creates a reverse lookup map for table
* regionName_columnFamilyName_storeFileName to the full Path. <br>
* Example...<br>
* Key = 3944417774205889744 <br>
* Value = hdfs://localhost:51169/user/userid/-ROOT-/70236052/info/3944417774205889744
* Key = regionName_columnFamilyName_storeFileName<br>
* Value = /hbase/data/namespaceName/tableName/regionName/columnFamilyName/storeFileName<br>
* @param fs The file system to use.
* @param hbaseRootDir The root directory to scan.
* @param sfFilter optional path filter to apply to store files
Expand Down Expand Up @@ -1242,14 +1242,14 @@ private Map<String, Path> getTableStoreFilePathMap(final FileSystem fs, final Pa
}

/**
* Runs through the HBase rootdir/tablename and creates a reverse lookup map for table StoreFile
* names to the full Path. Note that because this method can be called on a 'live' HBase system
* that we will skip files that no longer exist by the time we traverse them and similarly the
* user of the result needs to consider that some entries in this map may not exist by the time
* this call completes. <br>
* Runs through the HBase rootdir/tablename and creates a reverse lookup map for table
* regionName_columnFamilyName_storeFileName to the full Path. Note that because this method can
* be called on a 'live' HBase system that we will skip files that no longer exist by the time we
* traverse them and similarly the user of the result needs to consider that some entries in this
* map may not exist by the time this call completes. <br>
* Example...<br>
* Key = 3944417774205889744 <br>
* Value = hdfs://localhost:51169/user/userid/-ROOT-/70236052/info/3944417774205889744
* Key = regionName_columnFamilyName_storeFileName<br>
* Value = /hbase/data/namespaceName/tableName/regionName/columnFamilyName/storeFileName<br>
* @param resultMap map to add values. If null, method will create and populate one to return
* @param fs The file system to use.
* @param hbaseRootDir The root directory to scan.
Expand Down Expand Up @@ -1290,6 +1290,7 @@ private static Map<String, Path> getTableStoreFilePathMap(Map<String, Path> resu

for (FileStatus regionDir : regionDirs) {
final Path dd = regionDir.getPath();
String regionName = dd.getName();

if (!exceptions.isEmpty()) {
break;
Expand All @@ -1312,7 +1313,8 @@ public void run() {
}
for (FileStatus familyDir : familyDirs) {
Path family = familyDir.getPath();
if (family.getName().equals(HConstants.RECOVERED_EDITS_DIR)) {
String familyName = family.getName();
if (familyName.equals(HConstants.RECOVERED_EDITS_DIR)) {
continue;
}
// now in family, iterate over the StoreFiles and
Expand All @@ -1321,7 +1323,13 @@ public void run() {
for (FileStatus sfStatus : familyStatus) {
Path sf = sfStatus.getPath();
if (sfFilter == null || sfFilter.accept(sf)) {
regionStoreFileMap.put(sf.getName(), sf);
// Only using the values in the map in offlineReferenceFileRepair and
// offlineHLinkFileRepair, so the outside logic is same.
// If the file in parent region lost, but the two daughters region both have
// reference to the file, need sideline both daughter region's reference file
// (the two file has same file name, see the logic
// in org.apache.hadoop.hbase.regionserver.HRegionFileSystem.splitStoreFile).
regionStoreFileMap.put(regionName + "_" + familyName + "_" + sf.getName(), sf);
}
}
}
Expand Down