Merge pull request #1667 from saulpw/v2.11

V2.11 Release

.github/workflows/main.yml

@@ -14,7 +14,7 @@ jobs:
 
     strategy:
       matrix:
-        python-version: [3.6, 3.7, 3.8, 3.9, "3.10"]
+        python-version: [3.7, 3.8, 3.9, "3.10", "3.11"]
 
     runs-on: ubuntu-latest
     steps:
@@ -24,9 +24,6 @@ jobs:
       uses: actions/setup-python@v2
       with:
         python-version: ${{ matrix.python-version }}
-
-    - name: Create .visidatarc
-      run: touch ~/.visidatarc
 
     - name: Set up Environment
       run: |
@@ -52,3 +49,16 @@ jobs:
       run: dev/test.sh
       shell: bash
 
+    - name: Test macros
+      run: |
+        mkdir -p ~/.visidata
+        cp tests/macros/macro* ~/.visidata/
+        vd --batch -p tests/macros/test_macro.vd --output tests/macros/golden/test_macro.tsv
+        git --no-pager diff --numstat tests/
+        git --no-pager diff --exit-code tests/
+
+    - name: Check docs are built without errors
+      run: |
+        sudo apt install aha
+        dev/mkman.sh
+

CHANGELOG.md

@@ -1,6 +1,67 @@
 # VisiData version history
 
-# v2.10.2 (2022-10-XX)
+# v2.11 (2023-01-15)
+
+- [ci] drop support for Python 3.6 (related to https://github.com/actions/setup-python/issues/543)
+- [ci] add support for Python 3.11 (#1585)
+
+- [dirsheet] add `open-dir-parent` (bound to backtick)
+- [join] add new "concat" jointype to behave similar to "append" but keeps first sheet type and columns (requested by @frosencrantz #1598)
+- [zip] add multisave for `.zip` (save each sheet in options.save_filetype format into given .zip file)
+- [sysedit] add `sysedit-selected` (bound to `g Ctrl+O`) (requested by @Delapouite #1596)
+    - edit cells in multiple rows in `$EDITOR`
+    - only handles cell modifications, not added or deleted rows
+
+
+## Improvements
+
+- [aggregators] add 95 and 99 percentile (p95 and p99)
+- [fill-col] speed up `fill-col` for sheets with many empty cells (PR by @midichef #1657)
+- [loaders hdf5] add support for arrays of scalars (requested by @linwaytin #1602)
+- [graph] fail if no numeric xcols are given
+- [open-cell-file] warn when file or url in cell does not exist (requested by @geekscrapy #1540)
+- [sqlite] add passthrough options (reported by @cwarden #1622)
+- [sqlite] add options.sqlite_onconnect to be executed before running any statement (requested by @cwarden #1622)
+- [xml] add passthrough options for xml_parser; default xml_parser_huge_tree=True (PR by @midichef #1668)
+
+## Bugfixes
+
+- [columns] `dup-sheet` now carries over attributes of columns added by `add-column`
+- [columns] **SettableColumn** should not be deferred (reported by @frosencrantz #1568)
+- [customdate] recognise type-customdate as numeric (requested by @tdussa #1613)
+- [describe] fix custom describe aggregators (reported by @edupont #1574)
+- [dirsheet] fix incorrect filename with multiple extensions (reported by @kunliugithub #1571)
+- [display] show `disp_oddspace` for surrogate escapes (reported by @geekscrapy #1544)
+- [graph] fix div-by-zero with only one y-value (reported by @midichef #1673)
+- [install] ensure setuptools files have appropriate permissions (reported by @icp1994 #1591)
+- [install] update data files in setup.py based on PEP 420 (reported by @Oblomov #1675)
+- [keystrokes] add `kDN` and `kUP` to translation table (reported by @djpohly #1336)
+- [loaders html] fix loading of relative links in html table (reported by @frosencrantz #1599)
+- [loaders xlsx] store `None` as empty string in `save_xlsx` (reported and PR by @dbaynard #1626 #1629)
+- [macros] override CLI parsing options for MacrosSheet (reported by @frosencrantz #1607)
+- [macros] query again for keystroke if used by existing macro (#1658)
+- [macros] do not include `nonLogged` commands in macro (reported by @geekscrapy #1569)
+- [macros] add reload for **MacroSheet** (reported by @geekscrapy #1569)
+- [menu] 2x ESC should exit menu
+- [mouse] fix mouse-clicks on statusbar when splitpane is off (reported by @frosencrantz #1625)
+- [numpy] fix loader
+- [open_txt] fix Exception with `open-config` when no `~/.visidatarc` (reported by @gunchev #1611)
+- [pdb] fix entering of pdb breakpoints for Python 3.9+ (reported by @jasonwirth #1317)
+- [sheets] sort all sheets on global **Sheets Sheet** (reported by @franzhuang #1620)
+- [types] format int/vlen as true int (reported by @xlucn #1674)
+- [unzip-http] fix file extraction (`x`) on remote zip file
+- [unzip-http] handle files smaller than 64K (reported by @frosencrantz #1567)
+- [zsh-completion] fixed (reported by @pigmonkey #1583; PR by @Freed-Wu #1646)
+
+## API
+
+- raise Exception from causes in utils.py (PR by @cool-RR #1633)
+- add `HistogramColumn` to allow overrides (requested by @andycraig #1621)
+- easier external numeric types with `@vd.numericType()` decorator (inspired by @s1291 #1394)
+
+- [frequency table] `dive-rows` renamed to `dive-selected`
+
+# v2.10.2 (2022-10-08)
 
 - add .vdx, a simplified new cmdlog format
 - add `-N`/`--nothing` command to disable loading .visidatarc and plugin addons

README.md

@@ -1,5 +1,10 @@
 
-# VisiData v2.10.2 [![twitter @VisiData][1.1]][1] [![CircleCI](https://circleci.com/gh/saulpw/visidata/tree/stable.svg?style=svg)](https://circleci.com/gh/saulpw/visidata/tree/stable) [![Gitpod ready-to-code](https://img.shields.io/badge/Gitpod-ready--to--code-blue?logo=gitpod)](https://gitpod.io/#https://github.com/saulpw/visidata)
+# VisiData v2.11
+
+[![twitter @VisiData][1.1]][1]
+[![Tests](https://github.com/saulpw/visidata/workflows/visidata-ci-build/badge.svg)](https://github.com/saulpw/visidata/actions/workflows/main.yml)
+[![Gitpod ready-to-code](https://img.shields.io/badge/Gitpod-ready--to--code-blue?logo=gitpod)](https://gitpod.io/#https://github.com/saulpw/visidata)
+[![discord](https://img.shields.io/discord/880915750007750737?label=discord)](https://visidata.org/chat)
 
 A terminal interface for exploring and arranging tabular data.
 
@@ -10,7 +15,7 @@ VisiData supports tsv, csv, sqlite, json, xlsx (Excel), hdf5, and [many other fo
 ## Platform requirements
 
 - Linux, OS/X, or Windows (with WSL)
-- Python 3.6+
+- Python 3.7+
 - additional Python modules are required for certain formats and sources
 
 ## Install

dev/checklists/manual-tests.md

@@ -58,7 +58,7 @@
             - columns sheet should remain where it is, and the source sheet below gets moved to the top of the other window
         - redo with opening the columns sheet on the other
             - bug
-                - when Shift+Z is done on bottom pane, its second sheet becomes the second sheet of the top stack, isntead of the first sheet of the top stack
+                - when Shift+Z is done on bottom pane, its second sheet becomes the second sheet of the top stack, instead of the first sheet of the top stack
         - test 3
             - Shift+Z with only one file
             - Shift+Z with two panes, each pane's stack only has one file

dev/checklists/release.md

@@ -71,7 +71,8 @@
 
   Push to pypi
     ```
-    python3 setup.py sdist bdist_wheel --universal
+    python3 setup.py sdist bdist_wheel
+    chmod -R a+rX .
     twine upload dist/*
     ```
 

dev/design/176-miscrules.md

@@ -1,4 +1,4 @@
-# Miscellanous rules discovered during implementation
+# Miscellaneous rules discovered during implementation
 
 - Don't put an @asynccache on calcValue, unless you know the rowdef to be simple.  asynccache will expand the entire row every time it is called, which can be more expensive than the actual calcValue.