barroso_flag_duplicates

Flag or Remove Duplicate Specimens

barRoso::barroso_flag_duplicates()

Description

Identifies and optionally removes duplicate herbarium specimen records based on collector name and number, or—when the number is missing—by species name, collector and date. Duplicates are grouped into blocks, and blocks with conflicting identifications are highlighted. The herbaria and catalog numbers of all duplicates are stored together, so no information is lost when duplicates are removed.

Details

This function is part of the internal workflow of the barRoso package, supporting record reconciliation and dataset cleaning. Duplicates are records with the same collector (recordedBy) and collector number (recordNumber). For records without collector number, duplicates must share the species name, the collector and the full collection date (year, month, day). Records with unknown collector (NA or "Unknown") are never treated as duplicates.

The following columns are added after the collector number:

duplicate: TRUE for records belonging to a block of duplicates.
duplicateGroup: an identifier of each block of duplicates.
identificationConflict: TRUE when the duplicates of a block are identified as different species (e.g. "Ormosia arborea" and "Ormosia fastigiata") or genera. A duplicate identified just to genus (e.g. "Ormosia") does not conflict with a complete name of the same genus. Names are compared exactly, so spelling variants (e.g. "Ormosia trifoliata" and "Ormosia trifoliolata") are also flagged as conflicts, which helps finding misspelled names.
duplicateIdentifications: all names found in the block, e.g. "Ormosia arborea | Ormosia fastigiata".
duplicateCollectionCodes: all herbaria of the block, e.g. "MO | JBB".
duplicateCatalogNumbers: all catalog numbers of the block with their herbarium, e.g. "MO 2839102 | JBB 13548".

The specific epithet is taken from specificEpithet or, when missing, from a binomial in the column species (as in GBIF downloads).

When rm_duplicates = TRUE, one record per block is kept, chosen as follows:

Records identified to species are preferred over those identified just to genus.
Among them, records with the most frequent identification in the block are preferred (e.g. the name given by two duplicates rather than by one). When names are equally frequent, all of them go to the next rule.
Among them, records with latitude and longitude, and then records with more filled columns, are preferred. If records are still tied, the first one in the data is kept.

The kept record still has identificationConflict and all the names, herbaria and catalog numbers of its block, so conflicting identifications remain visible after removing the duplicates.

Arguments

Argument	Description
df	A data frame with biodiversity specimen records.
rm_duplicates	Logical; if `TRUE`, keeps just one record per block of duplicates (default: `FALSE`).
colname_recordedBy	Column name for the collector (default: `"recordedBy"`).
colname_recordNumber	Column name for the collector number (default: `"recordNumber"`).
colname_genus	Column name for the genus (default: `"genus"`).
colname_specificEpithet	Column name for the specific epithet (default: `"specificEpithet"`).
colname_collectionCode	Column name for the herbarium acronym (default: `"collectionCode"`).
colname_catalogNumber	Column name for the catalog number (default: `"catalogNumber"`).
colname_decimalLatitude	Column name for latitude (default: `"decimalLatitude"`).
colname_decimalLongitude	Column name for longitude (default: `"decimalLongitude"`).

Value

A data frame with the duplicate columns described above. If rm_duplicates = TRUE, only one record per block of duplicates is kept.

Examples

df <- read.csv("herbarium_data.csv")
df_flagged <- barroso_flag_duplicates(df)

# Blocks of duplicates with conflicting identifications
df_flagged[df_flagged$identificationConflict %in% TRUE, ]

df_clean <- barroso_flag_duplicates(df, rm_duplicates = TRUE)

--- title: 'barroso_flag_duplicates' description: 'Flag or Remove Duplicate Specimens' toc: true toc-depth: 3 --- ```{r} #| eval: false barRoso::barroso_flag_duplicates() ``` ### Description Identifies and optionally removes duplicate herbarium specimen records based on collector name and number, or—when the number is missing—by species name, collector and date. Duplicates are grouped into blocks, and blocks with conflicting identifications are highlighted. The herbaria and catalog numbers of all duplicates are stored together, so no information is lost when duplicates are removed. ### Details This function is part of the internal workflow of the `barRoso` package, supporting record reconciliation and dataset cleaning. Duplicates are records with the same collector (`recordedBy`) and collector number (`recordNumber`). For records without collector number, duplicates must share the species name, the collector and the full collection date (`year`, `month`, `day`). Records with unknown collector (`NA` or `"Unknown"`) are never treated as duplicates. The following columns are added after the collector number: - `duplicate`: `TRUE` for records belonging to a block of duplicates. - `duplicateGroup`: an identifier of each block of duplicates. - `identificationConflict`: `TRUE` when the duplicates of a block are identified as different species (e.g. `"Ormosia arborea"` and `"Ormosia fastigiata"`) or genera. A duplicate identified just to genus (e.g. `"Ormosia"`) does not conflict with a complete name of the same genus. Names are compared exactly, so spelling variants (e.g. `"Ormosia trifoliata"` and `"Ormosia trifoliolata"`) are also flagged as conflicts, which helps finding misspelled names. - `duplicateIdentifications`: all names found in the block, e.g. `"Ormosia arborea | Ormosia fastigiata"`. - `duplicateCollectionCodes`: all herbaria of the block, e.g. `"MO | JBB"`. - `duplicateCatalogNumbers`: all catalog numbers of the block with their herbarium, e.g. `"MO 2839102 | JBB 13548"`. The specific epithet is taken from `specificEpithet` or, when missing, from a binomial in the column `species` (as in GBIF downloads). When `rm_duplicates = TRUE`, one record per block is kept, chosen as follows: 1. Records identified to species are preferred over those identified just to genus. 2. Among them, records with the most frequent identification in the block are preferred (e.g. the name given by two duplicates rather than by one). When names are equally frequent, all of them go to the next rule. 3. Among them, records with latitude and longitude, and then records with more filled columns, are preferred. If records are still tied, the first one in the data is kept. The kept record still has `identificationConflict` and all the names, herbaria and catalog numbers of its block, so conflicting identifications remain visible after removing the duplicates. ### Arguments | Argument | Description | |---|---| | df | A data frame with biodiversity specimen records. | | rm_duplicates | Logical; if `TRUE`, keeps just one record per block of duplicates (default: `FALSE`). | | colname_recordedBy | Column name for the collector (default: `"recordedBy"`). | | colname_recordNumber | Column name for the collector number (default: `"recordNumber"`). | | colname_genus | Column name for the genus (default: `"genus"`). | | colname_specificEpithet | Column name for the specific epithet (default: `"specificEpithet"`). | | colname_collectionCode | Column name for the herbarium acronym (default: `"collectionCode"`). | | colname_catalogNumber | Column name for the catalog number (default: `"catalogNumber"`). | | colname_decimalLatitude | Column name for latitude (default: `"decimalLatitude"`). | | colname_decimalLongitude | Column name for longitude (default: `"decimalLongitude"`). | ### Value A data frame with the duplicate columns described above. If `rm_duplicates = TRUE`, only one record per block of duplicates is kept. ### Examples ```r df <- read.csv("herbarium_data.csv") df_flagged <- barroso_flag_duplicates(df) # Blocks of duplicates with conflicting identifications df_flagged[df_flagged$identificationConflict %in% TRUE, ] df_clean <- barroso_flag_duplicates(df, rm_duplicates = TRUE) ```