Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Harvesting : Add granularity to ListRecords when using from parameter #11038

Open
wants to merge 2 commits into
base: develop
Choose a base branch
from
Open

Harvesting : Add granularity to ListRecords when using from parameter #11038

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
284ea30
#11020 - Harvesting : Now performs an Identify request before a ListR…
luddaniel Nov 20, 2024
ddab296
add release note
luddaniel Nov 20, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions doc/release-notes/11038-unconsidered-harvesting-granularity.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
Bug Fix:
OAI Client harvesting now uses the correct granularity while re-run a partial harvest (using the `from` parameter). The correct granularity comes from the `Identify` verb request.
14 changes: 10 additions & 4 deletions src/main/java/edu/harvard/iq/dataverse/harvest/client/oai/OaiHandler.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@
import io.gdcc.xoai.model.oaipmh.results.record.Header;
import io.gdcc.xoai.model.oaipmh.results.MetadataFormat;
import io.gdcc.xoai.model.oaipmh.results.Set;
import io.gdcc.xoai.model.oaipmh.verbs.Identify;
import io.gdcc.xoai.serviceprovider.ServiceProvider;
import io.gdcc.xoai.serviceprovider.exceptions.BadArgumentException;
import io.gdcc.xoai.serviceprovider.exceptions.InvalidOAIResponse;
Expand Down Expand Up @@ -289,6 +290,8 @@ private ListIdentifiersParameters buildListIdentifiersParams() throws OaiHandler
mip.withMetadataPrefix(metadataPrefix);

if (this.fromDate != null) {
Identify identify = runIdentify();
mip.withGranularity(identify.getGranularity().toString());
mip.withFrom(this.fromDate.toInstant());
}

Expand All @@ -311,10 +314,13 @@ public String getProprietaryDataverseMetadataURL(String identifier) {
return requestURL.toString();
}

public void runIdentify() {
// not implemented yet
// (we will need it, both for validating the remote server,
// and to learn about its extended capabilities)
public Identify runIdentify() throws OaiHandlerException {
ServiceProvider sp = getServiceProvider();
try {
return sp.identify();
} catch (InvalidOAIResponse ior) {
throw new OaiHandlerException("No valid response received from the OAI server.");
}
}

public Map<String,String> makeCustomHeaders(String headersString) {
Expand Down
Loading