Problem/Motivation

According to Wikipedia: RIS (file format), several pairs of keys are synonymous:

  • A1 and AU (primary author)
  • A2 and ED (secondary author or editor)
  • AB and N2 (abstract)
  • PY and Y1 (publication year) - maybe also YR
  • T1 and TI (primary title, not described as synonyms)

If an .ris file has one or more A1 keys, for example, then they should be imported as authors (Contributor entities).

Steps to reproduce

  1. Install Drupal with the bibcite_import, bibcite_ris modules and their dependencies.
  2. From /admin/content/bibcite/reference/import, import the RIS file below.
  3. Expected result: each imported Reference has a title and at least one author.
  4. Actual result: all three imported References have neither title nor author.

Proposed resolution

In Drupal\bibcite_ris\Encoder\RISEncoder::decode(), I already see the code

      // Year of publication.
      if (isset($record['Y1']) && !isset($fields['Y1']) && !isset($records['PY'])) {
        $record['PY'] = $record['Y1'];
        unset($record['Y1']);
      }

Add similar handling for the other pairs of synonyms.

Remaining tasks

  1. Add test coverage.

User interface changes

None

API changes

None

Data model changes

None

Issue fork bibcite-3584018

Command icon Show commands

Start within a Git clone of the project using the version control instructions.

Or, if you do not have SSH keys set up on git.drupalcode.org:

Comments

benjifisher created an issue. See original summary.

benjifisher’s picture

Issue summary: View changes
benjifisher’s picture

Looking at the code in Drupal\bibcite_ris\Encoder\RISEncoder::decode(), I see a lot of problems.

Start with the snippet I already quoted:

      // Year of publication.
      if (isset($record['Y1']) && !isset($fields['Y1']) && !isset($records['PY'])) {
        $record['PY'] = $record['Y1'];
        unset($record['Y1']);
      }

It should be $record['PY'] (singular), not $records['PY']. That is clearly a mistake.

With the default configuration, isset($fields['Y1']) (and similar) will always be TRUE: in bibcite_entity.mapping.ris.yml, Y1 is mapped to ''. So this is dead code.

There are several blocks like this one:

      // Secondary title.
      if ($this->checkKeys(['JA', 'JF', 'T2'], $record) && $record['JA'] === $record['JF'] && $record['JA'] === $record['T2']) {
        unset($record['JA']);
        unset($record['JF']);
      }

In other words, if there is a high degree of redundancy, then remove it. But this does not help at all if the expected key is not set, but one of the alternatives (aliases) is.

I think I am going to radically change this function. We will see what happens to the existing test coverage, and then we will want to add some new tests.

benjifisher’s picture

Assigned: benjifisher » Unassigned
Status: Active » Needs review
benjifisher’s picture

Issue summary: View changes
Issue tags: +Needs tests

mark_fullmer made their first commit to this issue’s fork.

mark_fullmer’s picture

Status: Needs review » Needs work

I started adding test coverage for the synonmous keys. Mostly things look fine, but in testing, I believe I established that Bibcite RIS import is currently not importing editor values at all (either with the "A2" or "ED" keys). I confirmed this by testing on the 3.0.x branch, too.

I am happy to treat that as a separate issue than this one, as this issue is really about handling synomous keys. Marking this as "Needs work", and either we omit testing for the Editor key until it's supported, or we keep this issue on hold until the Editor key is supported.

mark_fullmer’s picture

Support for secondary author (editor) data in RIS format is staged in #3585043: RIS format does not import "Secondary Author" (Editor) data.

benjifisher’s picture

@mark_fullmer:

I can probably fix the missing editors as part of this issue.

The current version is good enough for my current purposes, and for the module with default configuration, but I think it should work harder to respect non-default configuration.

Suppose some site owner noticed that all their RIS files used Y1 instead of PY. Then they might update bibcite_entity.mapping.ris to map Y1 to bibcite_year and un-map PY. The current MR will unset($record['Y1']), so it will not import anything for bibcite_year.

What do you think is the right thing to do? If we want to support all the aliases with standard configuration, then we should pay attention to all of them, even if they are un-mapped in bibcite_entity.mapping.ris. We should certainly make sure that the "last man standing" is one of the mapped keys. But maybe we should ignore the keys that are not mapped.

I guess I am leaning to the idea of looking at all of them, mapped or not, and setting whichever one will be used based on the configuration.

benjifisher’s picture

Running the test locally, I noticed a deprecation notice, so I opened #3585052: [Drupal 11] The supportedInterfaceOrClass property is deprecated. It should be pretty easy to fix.

P.S. I am glad that I did that, since otherwise I would have duplicated the effort you just made.

mark_fullmer’s picture

I can probably fix the missing editors as part of this issue.

Ships passing in the night! This should be taken care of in https://git.drupalcode.org/project/bibcite/-/merge_requests/60/diffs?com...

I guess I am leaning to the idea of looking at all of them, mapped or not, and setting whichever one will be used based on the configuration.

Yes, this is an interesting decision point. This proposal makes sense to me, and I think we can assume that we don't need to support a scenario where two keys for the same thing have been defined in one file (e.g., someone has a Y1 and a PY key in the same entry).

benjifisher’s picture

I updated the MR, implementing the change discussed in Comments #10 and #12.

Some tests are failing. I will have to look into it, maybe not until next week.

One thing I do not understand: the author and editor fields seem to be imported even if no RIS fields are mapped to these fields in bibcite_entity.mapping.ris.

mark_fullmer’s picture

One thing I do not understand: the author and editor fields seem to be imported even if no RIS fields are mapped to these fields in bibcite_entity.mapping.ris.

I think this is due to a couple factors. First, /modules/bibcite_ris/config/install/bibcite_entity.mapping.ris.yml provides a default mapping for AU: author, so even if no one has set a custom mapping, that field will be mapped if the RIS file has an AU value. (Unless I'm misunderstanding and you're saying that it is mapped even if someone has *unset* the default mapping at admin/config/bibcite/settings/mapping/ris).

Second, the recent work to mimic how the BibtexReferenceNormalizer maps "editor" contributors for RIS in #3585043: RIS format does not import "Secondary Author" (Editor) data results in the "ED" key always being eligible for populating contributors of type "editor," regardless of mapping settings.

Reflecting on the original implementation by a previous maintainer in #3116100: Bibtex Encoder disregards `editor` key for use as Author with editor role when importing Bibtex content and my replication of that business logic, I feel like this is both opaque and not ideally flexible. If we wanted "ideal," this would probably involve reworking ReferenceNormalizerBase::getContributorKey to look for multiple keys that are considered by Bibcite to be authors/editors of some sort.

I don't think that work needs to be part of this issue, however.

benjifisher’s picture

@mark_fullmer:

Along with changing ReferenceNormalizerBase::getContributorKey(), we might have to change the structure of the configuration. I added #3585875: Re-think how this module uses bibcite_entity.mapping.* configuration, and I am linking it as a related issue.

I am glad you mentioned that (and that it is out of scope for this issue). It feels like a kluge to be hard-coding all of these keys that show up in the bibcite_entity.mapping.* configuration. I will not worry about it for now, and maybe I will find time to work on it on that new issue.

I think I can fix the problems I mentioned in Comment #13 with a few more lines of code, and I will try to add test coverage by the end of the week.

Another subject: there are a lot of Fixed issues in the queue for this module, and the ones I have checked do not have anyone credited in their contribution records. That means no one is getting issue credit for this work. See Granting credit to issue contributors. In part,

As a project maintainer, it is your responsibility, when marking an issue as Fixed or closed, to grant credit to people who actually helped resolve the issue. Only project maintainers can grant credit for issues in a project.

mark_fullmer’s picture

Another subject: there are a lot of Fixed issues in the queue for this module, and the ones I have checked do not have anyone credited in their contribution records.

Thanks! Can you give me an example of a contribution record where you're not seeing the people involved credited? I have been adding credit after marking issues as fixed, but I may be doing it wrong. In the two examples below, it looks to me like the people are credited. Are you seeing something different?

- https://new.drupal.org/contribution-record/10052355
- https://new.drupal.org/contribution-record/11449739

benjifisher’s picture

@mark_fullmer:

I am very sorry. I did not re-check.

I noticed this several days ago ... maybe a week or two. Every issue I checked had an empty contribution record.

Either I was very confused or maybe I checked on a day when d.o was recovering from some recent turmoil (server relocation, data migration). I hope it is the latter, but I cannot be sure.

mark_fullmer’s picture

No worries! I appreciate the second set of eyes!

benjifisher’s picture

Assigned: benjifisher » Unassigned
Status: Needs work » Needs review

I added some test coverage, so this issue is ready for review.

In the process, I discovered a couple of problems with the code, which I fixed. I do not claim to follow TDD, but this was in that spirit.

I got some help from Claude Code to set up the test. The problems only came to light after I started tweaking the test "manually".

mark_fullmer’s picture

Status: Needs review » Fixed

Okay, the final changes look good to me. Thanks, also for adding more comprehensive test coverage. We have follow-up issues in #3057695: Make it possible to map from several format fields to reference's Authors field and #3585875: Re-think how this module uses bibcite_entity.mapping.* configuration to consider addressing more completeness in how Bibcite can handle contributors and mappings, though as a maintainer, I think it make sense to not prioritize those over more urgent bugs, especially given that no one has yet asked for those enhancements. (Don't want to make "ideally" the enemy of "sufficiently"!)

Now that this issue is closed, review the contribution record.

As a contributor, attribute any organization that helped you, or if you volunteered your own time.

Maintainers, credit people who helped resolve this issue.

benjifisher’s picture

@mark_fullmer:

No argument here!

Thanks for working with me on this and several other issues. I am glad that you made a new release with all these fixes. Now we can upgrade and drop a bunch of patches that we have been applying.

Status: Fixed » Closed (fixed)

Automatically closed - issue fixed for 2 weeks with no activity.