File and directory transfer

copy() copies files or directories (perhaps recursively) from one collection to another.

transfer() and transfer_item() are lower-level functions that allow for one or more files or directories, including symbolic links, to be transferred as a single task.

transfer_item() constructs the JSON description of a single transfer, e.g., of a source file to a destination file. It is used as input to transfer().

transfer_label() is a helper function to provide an identifying label for each task, visible for instance in the Globus web application.

Usage

copy(
  source,
  destination,
  source_path = "",
  destination_path = "",
  recursive = FALSE,
  ...
)

transfer(
  source,
  destination,
  transfer_items,
  label = transfer_label(),
  notify_on_succeeded = TRUE,
  notify_on_failed = TRUE,
  encrypt_data = FALSE,
  verify_checksum = FALSE,
  preserve_timestamp = FALSE,
  delete_destination_extra = FALSE,
  skip_source_errors = FALSE,
  fail_on_quota_errors = FALSE,
  filter_rules = NULL,
  sync_level = NULL
)

transfer_item(source_path, destination_path, recursive)

transfer_label()

Arguments

source

a tibble containing a single row with column id, the unique identifier for the source collection.

destination

a tibble containing a single row with column id, the unique identifier of the destination collection.

source_path

character(1) path from the source collection root to the directory or file to be copied.

destination_path

character(1) path from the destination collection root to the location of the directory or file to be copied.

recursive

logical(1) when TRUE and source_path is a directory, copy directory content recursively to destination_path. recursive = TRUE is an error when the source is a file.

...

additional arguments passed from copy() to transfer().

transfer_items

character() of transfer_item() or symlink_item() objects.

label

character(1) identifier for the task.

notify_on_succeeded

logical(1), see https://docs.globus.org/api/transfer/task_submit/#transfer_specific_fields.

notify_on_failed

logical(1).

encrypt_data

logical(1).

verify_checksum

logical(1).

preserve_timestamp

logical(1).

delete_destination_extra

logical(1).

skip_source_errors

logical(1).

fail_on_quota_errors

logical(1)

filter_rules

not implemented.

sync_level

not implemented.

Value

copy() and transfer() return a tibble with columns submission_id, task_id and code. code is one of 'Accepted' (the task is queued for execution) or Duplicate (the task is a re-submission of an existing task). Use the return value to check or cancel the task with task_status() or task_cancel().

transfer_label() returns a character(1) label including date, time, and user information, with suffix rglobus-transfer.

Details

copy() implements the common task of copying files or folders between collections. transfer() allows specification of multiple transfer operations in a single task.

copy() and transfer() submit a task to initiate a transaction, but it is necessary to check on the status of the task with task_status(). It is also necessary to manually cancel tasks that fail with task_cancel().

In the Globus documentation,

https://docs.globus.org/api/transfer/task_submit/#document_types

sections 2.2, 2.3, and 2.4 describe options relevant to copying directories and files.

https://docs.globus.org/api/transfer/task_submit/#errors

describes errors during task submission.

Examples

my_collection <- my_collections()
hubmap <-
    collections("HuBMAP Public") |>
    dplyr::filter(display_name == "HuBMAP Public")

##
## File and directory copying
##

## HubMAP dataset id, from HuBMAPR::datasets()
dataset_id <- "d1dcab2df80590d8cd8770948abaf976"
globus_ls(hubmap, dataset_id)
#> # A tibble: 5 × 4
#>   name                                          last_modified         size type 
#>   <chr>                                         <chr>                <int> <chr>
#> 1 extras                                        2023-01-12 15:32:24…  4096 dir  
#> 2 imzML                                         2023-01-04 17:56:12…  4096 dir  
#> 3 ometiffs                                      2023-01-04 17:58:26…  4096 dir  
#> 4 d1dcab2df80590d8cd8770948abaf976-metadata.tsv 2023-01-12 15:32:21…  1901 file 
#> 5 metadata.json                                 2024-04-17 23:24:32… 34814 file 

## copy file from HuBMAP dataset to `my_collection`
source_path <- paste0(dataset_id, "/metadata.json")
destination_path <- paste0("tmp/HuBMAP/", source_path)
task <- copy(
    hubmap, my_collection,
    source_path, destination_path,
    notify_on_succeeded = FALSE
)
task
#> # A tibble: 1 × 3
#>   submission_id                        task_id                             code 
#>   <chr>                                <chr>                               <chr>
#> 1 81c574af-5a81-11ef-9082-a128da25f366 81c574ae-5a81-11ef-9082-a128da25f3… Acce…

task_status(task)
#> # A tibble: 1 × 5
#>   task_id                              type     status nice_status label        
#>   <chr>                                <chr>    <chr>  <chr>       <chr>        
#> 1 81c574ae-5a81-11ef-9082-a128da25f366 TRANSFER ACTIVE Queued      2024-08-14 1…

## default task label
transfer_label()
#> [1] "2024-08-14 17:09:34-0400 ma38727 rglobus-transfer"