Is saving to .saw format still supported?

[apostolos-geyer]

Is saving in .saw format still supported?
I was trying to do so as was detailed on an old blog post here:
String dbName = tornadoes.save("/tmp/tablesaw/testdata"); Table tornadoes = Table.readTable(dbName);
However, when I try to do something similar
String dbName = byPHU.save("Positive-Cases-by-Public-Health-Unit"); (I didn't specify a path as I assume it will just save it into the project directory), Netbeans IDE gives me the warning:

I was looking to save it to .saw as using .read().csv() takes a solid 20-30 seconds for my dataset and, according to that blog post

If you use a large table more than once, you may want to save it in Tablesaw’s compressed columnar “.saw” format. In .saw format, reads and writes are (at least) an order of magnitude faster than the equivalent CSV operations, and disk usage is also greatly reduced.

I combed through the documentation and couldn't find any solutions, however I may have just missed it, sorry if that's the case. If the method no longer exists, is there a new one that I'm missing, or a different means of speeding up the initialization of large tables?

Thanks in advance.

[lwhite1]

I would recommend not using it as it's probably not entirely stable. You can usually save considerable time by providing the column types in the builder, since it will skip the type determination process.

…

On Wed, May 26, 2021 at 1:07 PM pauljamesgeyer ***@***.***> wrote: Is saving in .saw format still supported? I was trying to do so as was detailed on an old blog post here <https://jtablesaw.wordpress.com/user-guide/tables/>: String dbName = tornadoes.save("/tmp/tablesaw/testdata"); Table tornadoes = Table.readTable(dbName); However, when I try to do something similar String dbName = byPHU.save("Positive-Cases-by-Public-Health-Unit"); (I didn't specify a path as I assume it will just save it into the project directory), Netbeans IDE gives me the warning: [image: Screen Shot 2021-05-26 at 12 59 24 PM] <https://user-images.githubusercontent.com/79337131/119701316-34447800-be22-11eb-8947-30de77fdfff1.png> I was looking to save it to .saw as using .read().csv() takes a solid 20-30 seconds for my dataset and, according to that blog post If you use a large table more than once, you may want to save it in Tablesaw’s compressed columnar “.saw” format. In .saw format, reads and writes are (at least) an order of magnitude faster than the equivalent CSV operations, and disk usage is also greatly reduced. I combed through the documentation and couldn't find any solutions, however I may have just missed it, sorry if that's the case. If the method no longer exists, is there a new one that I'm missing, or a different means of speeding up the initialization of large tables? Thanks in advance. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#950>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AA2FPAS7ZHW2PCVSBJ5MR6LTPUTFVANCNFSM45SPZA4A> .

[apostolos-geyer]

Thanks, I actually did end up digging into the source and figuring out how to export and import to .saw, and it's been working considerably well for me except for once in a while where it just doesn't load the table, however, all I have to do is restart the program and it runs fine.

What I'm trying to do is implement some form of object persistence for a class I'm building on top of Tablesaw.

public class Dataset{
    Table table;
    String name;
    String csvPath;
    String sawPath;
    String url;
    String lastUpdated;

I've been using XMLEncoder and XMLDecoder to serialize the String attributes so that Dataset's can be rebuilt easily and updated from the internet. Being the amatuer that I am, I couldn't figure out a way to serialize a ColumnType[] so I opted to just write to .saw to quickly rebuild the tables on startup because my .csv's are fairly large. Here are two examples of datasets I'm using:

Confirmed Covid Cases In Ontario by Public Health Unit: 
531459 rows X 18 cols
[Row_ID, Accurate_Episode_Date, Case_Reported_Date, Test_Reported_Date, Specimen_Date, Age_Group, Client_Gender, Case_AcquisitionInfo, Outcome1, Outbreak_Related, Reporting_PHU_ID, Reporting_PHU, Reporting_PHU_Address, Reporting_PHU_City, Reporting_PHU_Postal_Code, Reporting_PHU_Website, Reporting_PHU_Latitude, Reporting_PHU_Longitude]
[INTEGER, LOCAL_DATE, LOCAL_DATE, LOCAL_DATE, LOCAL_DATE, STRING, STRING, STRING, STRING, STRING, INTEGER, STRING, STRING, STRING, STRING, STRING, DOUBLE, DOUBLE]

Ontario Public Health Unit Population Estimates & Projections 2016-2023 by Age and Sex: 
159250 rows X 7 cols
[Population CYear, PHU Code, PHU, Age, Age Grp (Inf,5yr), Sex Type Code, # People]
[INTEGER, INTEGER, STRING, INTEGER, STRING, STRING, INTEGER]

If you have any ideas I'd appreciate it. If github discussions are not the place for comments like this, my apologies. Very impressive library so far by the way!