Skip to content

ARROW-14258: [R] Warn if an SF column is made into a table#11361

Closed
jonkeane wants to merge 4 commits into
apache:masterfrom
jonkeane:ARROW-14258-sf-warn
Closed

ARROW-14258: [R] Warn if an SF column is made into a table#11361
jonkeane wants to merge 4 commits into
apache:masterfrom
jonkeane:ARROW-14258-sf-warn

Conversation

@jonkeane

@jonkeane jonkeane commented Oct 7, 2021

Copy link
Copy Markdown
Member

No description provided.

@github-actions

github-actions Bot commented Oct 7, 2021

Copy link
Copy Markdown

@github-actions

github-actions Bot commented Oct 7, 2021

Copy link
Copy Markdown

⚠️ Ticket has no components in JIRA, make sure you assign one.

Comment thread r/R/metadata.R

attempt_to_save_row_level <- getOption("arrow.preserve_row_level_metadata", FALSE) &&
is.list(x) && !inherits(x, "POSIXlt")
if (attempt_to_save_row_level) {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You could move the warning to an else if (inherits(x, c("sfc", "sf"))) below this, maybe that removes the complexity warning? Though maybe we should warn regardless of whether we're saving the metadata, just a different warning?

Comment thread r/R/metadata.R Outdated
"One of the columns given appears to be an `sfc` SF column. Due to their unique ",
"nature, these columns do not convert to Arrow well. We are working on ",
"better ways to do this, but in the interim we recommend converting any `sfc` ",
"columns to WKB (well-known binary) columns before using them with Arrow.",

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you suggest a function in sf that does this?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We could also talk about sfarrow here (or maybe just in the news, since it'll be easier to link to?)

Comment thread r/tests/testthat/test-metadata.R Outdated
)
withr::with_options(
list("arrow.preserve_row_level_metadata" = TRUE), {
withr::local_options(list("arrow.preserve_row_level_metadata" = TRUE))

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It looks like you're setting the option twice?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah yes, that was a mistake, fixed now.

Comment thread r/.lintr Outdated
object_length_linter = object_length_linter(40),
object_usage_linter = NULL, # R6 methods are flagged,
cyclocomp_linter = cyclocomp_linter(26) # TODO: reduce to default of 15
cyclocomp_linter = cyclocomp_linter(20) # TODO: reduce to default of 15

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This appears to trigger a new annotation

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oops, I got too eager that the metadata function was our only excessively cyclocomplex function. I'll bump this back up

@nealrichardson nealrichardson left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Two notes but otherwise LGTM

@jonkeane jonkeane closed this in 1fa1a2b Oct 13, 2021
@ursabot

ursabot commented Oct 13, 2021

Copy link
Copy Markdown

Benchmark runs are scheduled for baseline = c875c5d and contender = 1fa1a2b. 1fa1a2b is a master commit associated with this PR. Results will be available as each benchmark for each run completes.
Conbench compare runs links:
[Finished ⬇️0.0% ⬆️0.0%] ec2-t3-xlarge-us-east-2
[Finished ⬇️0.0% ⬆️0.0%] ursa-i9-9960x
[Finished ⬇️0.09% ⬆️0.04%] ursa-thinkcentre-m75q
Supported benchmarks:
ursa-i9-9960x: langs = Python, R, JavaScript
ursa-thinkcentre-m75q: langs = C++, Java
ec2-t3-xlarge-us-east-2: cloud = True

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants