Repository navigation
Write Excel output from compute - #95
Merged
Merged
Conversation
compute to an .xlsx path failed with "this.out is null": clojask-io's get-output-func has no Excel writer, so the workers called nil. With an Excel input the same-format branch picked the input's csv writer instead and would have written csv text into the .xlsx file, and the returned dataframe could not read the file either way. compute now collects the rows in memory for an .xlsx path (unless a custom :output function is given), writes them with clojask-io's write-excel to sheet Sheet1, and returns a dataframe read back with read-excel. An .xls path is rejected before the output file is deleted, since write-excel always writes the xlsx format.
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
computeto an.xlsxpath crashed withCannot invoke "clojure.lang.IFn.invoke(Object, Object)" because "this.out" is null: clojask-io'sget-output-funcreturns nil for Excel formats, and the workers called it. With an Excel input (read-excel), the "same format as the input" branch picked the input's csv writer instead and would have written csv text into the.xlsxfile. Either way the returned dataframe was built with(dataframe output-dir), which cannot read Excel.Change
.xlsxoutput path (and no custom:outputfunction),computecollects the rows in memory, as it does for anilpath, writes them withclojask-io.output/write-excelto sheetSheet1, and returns a dataframe read back withread-excel. Excel workbooks are built in memory anyway, so the output has to fit in memory; the docs say so..xlspath throws aTypeExceptionbeforeinit-filedeletes the existing output file, sincewrite-excelalways writes the xlsx format.docs/documentation.md: thecomputeoutput path and return value describe the formats.Tests
regression-testgainscompute-writes-xlsx(simple compute; group-by aggregate with:header false; an Excel input written back to.xlsx) andcompute-rejects-xls-output(throws, leaves the existing file). Both fail on2.x.x(2 errors, 1 failure) and pass here. Full suite: 62 tests, 312 assertions, 0 failures.Found while updating clojask-examples'
in_memoryexample, which works around this by computing in memory and callingwrite-excelitself.