diff --git a/404.html b/404.html index 496fe46f..139e4467 100644 --- a/404.html +++ b/404.html @@ -8,13 +8,13 @@ - - + + -

404

There's nothing here.
+ - + diff --git a/LICENSE.html b/LICENSE.html index 17ff1369..e0945ead 100644 --- a/LICENSE.html +++ b/LICENSE.html @@ -8,8 +8,8 @@ - - + +
Frictionless Standards
- + diff --git a/assets/css/0.styles.9133e7b5.css b/assets/css/0.styles.31ed7ec1.css similarity index 97% rename from assets/css/0.styles.9133e7b5.css rename to assets/css/0.styles.31ed7ec1.css index 1b4a3b75..11ce0bdc 100644 --- a/assets/css/0.styles.9133e7b5.css +++ b/assets/css/0.styles.31ed7ec1.css @@ -1 +1 @@ -.theme-default-content code{color:#476582;padding:.25rem .5rem;margin:0;font-size:.85em;background-color:rgba(27,31,35,.05);border-radius:3px}.theme-default-content code .token.deleted{color:#ec5975}.theme-default-content code .token.inserted{color:#1a98ff}.theme-default-content pre,.theme-default-content pre[class*=language-]{line-height:1.4;padding:1.25rem 1.5rem;margin:.85rem 0;background-color:#282c34;border-radius:6px;overflow:auto}.theme-default-content pre[class*=language-] code,.theme-default-content pre code{color:#fff;padding:0;background-color:transparent;border-radius:0}.custom-block .custom-block-title{font-weight:600;margin-bottom:-.4rem}.custom-block.danger,.custom-block.tip,.custom-block.warning{padding:.1rem 1.5rem;border-left-width:.5rem;border-left-style:solid;margin:1rem 0}.custom-block.tip{border-color:#42b983}.custom-block.warning{background-color:rgba(255,229,100,.3);border-color:#e7c000;color:#6b5900}.custom-block.warning .custom-block-title{color:#b29400}.custom-block.warning a{color:#2c3e50}.custom-block.danger{background-color:#ffe6e6;border-color:#c00;color:#4d0000}.custom-block.danger .custom-block-title{color:#900}.custom-block.danger a{color:#2c3e50}.custom-block.details{display:block;position:relative;border-radius:2px;margin:1.6em 0;padding:1.6em;background-color:#eee}.custom-block.details h4{margin-top:0}.custom-block.details figure:last-child,.custom-block.details p:last-child{margin-bottom:0;padding-bottom:0}.custom-block.details summary{outline:none;cursor:pointer}.arrow{display:inline-block;width:0;height:0}.arrow.up{border-bottom:6px solid #ccc}.arrow.down,.arrow.up{border-left:4px solid transparent;border-right:4px solid transparent}.arrow.down{border-top:6px solid #ccc}.arrow.right{border-left:6px solid #ccc}.arrow.left,.arrow.right{border-top:4px solid transparent;border-bottom:4px solid transparent}.arrow.left{border-right:6px solid #ccc}.theme-default-content:not(.custom){max-width:740px;margin:0 auto;padding:2rem 2.5rem}@media (max-width:959px){.theme-default-content:not(.custom){padding:2rem}}@media (max-width:419px){.theme-default-content:not(.custom){padding:1.5rem}}.table-of-contents .badge{vertical-align:middle}body,html{padding:0;margin:0;background-color:#fff}body{font-family:-apple-system,BlinkMacSystemFont,Segoe UI,Roboto,Oxygen,Ubuntu,Cantarell,Fira Sans,Droid Sans,Helvetica Neue,sans-serif;-webkit-font-smoothing:antialiased;-moz-osx-font-smoothing:grayscale;font-size:16px;color:#2c3e50}.page{padding-left:20rem}.navbar{z-index:20;right:0;height:4.6rem;background-color:#fff;box-sizing:border-box;border-bottom:1px solid #eaecef}.navbar,.sidebar-mask{position:fixed;top:0;left:0}.sidebar-mask{z-index:9;width:100vw;height:100vh;display:none}.sidebar{font-size:16px;background-color:#fff;width:20rem;position:fixed;z-index:10;margin:0;top:4.6rem;left:0;bottom:0;box-sizing:border-box;border-right:1px solid #eaecef;overflow-y:auto}.theme-default-content:not(.custom)>:first-child{margin-top:4.6rem}.theme-default-content:not(.custom) a:hover{text-decoration:underline}.theme-default-content:not(.custom) p.demo{padding:1rem 1.5rem;border:1px solid #ddd;border-radius:4px}.theme-default-content:not(.custom) img{max-width:100%}.theme-default-content.custom{padding:0;margin:0}.theme-default-content.custom img{max-width:100%}a{font-weight:500;text-decoration:none}a,p a code{color:#1a98ff}p a code{font-weight:400}kbd{background:#eee;border:.15rem solid #ddd;border-bottom:.25rem solid #ddd;border-radius:.15rem;padding:0 .15em}blockquote{font-size:1rem;color:#999;border-left:.2rem solid #dfe2e5;margin:1rem 0;padding:.25rem 0 .25rem 1rem}blockquote>p{margin:0}ol,ul{padding-left:1.2em}strong{font-weight:600}h1,h2,h3,h4,h5,h6{font-weight:600;line-height:1.25}.theme-default-content:not(.custom)>h1,.theme-default-content:not(.custom)>h2,.theme-default-content:not(.custom)>h3,.theme-default-content:not(.custom)>h4,.theme-default-content:not(.custom)>h5,.theme-default-content:not(.custom)>h6{margin-top:-4.1rem;padding-top:5.6rem;margin-bottom:0}.theme-default-content:not(.custom)>h1:first-child,.theme-default-content:not(.custom)>h2:first-child,.theme-default-content:not(.custom)>h3:first-child,.theme-default-content:not(.custom)>h4:first-child,.theme-default-content:not(.custom)>h5:first-child,.theme-default-content:not(.custom)>h6:first-child{margin-top:-1.5rem;margin-bottom:1rem}.theme-default-content:not(.custom)>h1:first-child+.custom-block,.theme-default-content:not(.custom)>h1:first-child+p,.theme-default-content:not(.custom)>h1:first-child+pre,.theme-default-content:not(.custom)>h2:first-child+.custom-block,.theme-default-content:not(.custom)>h2:first-child+p,.theme-default-content:not(.custom)>h2:first-child+pre,.theme-default-content:not(.custom)>h3:first-child+.custom-block,.theme-default-content:not(.custom)>h3:first-child+p,.theme-default-content:not(.custom)>h3:first-child+pre,.theme-default-content:not(.custom)>h4:first-child+.custom-block,.theme-default-content:not(.custom)>h4:first-child+p,.theme-default-content:not(.custom)>h4:first-child+pre,.theme-default-content:not(.custom)>h5:first-child+.custom-block,.theme-default-content:not(.custom)>h5:first-child+p,.theme-default-content:not(.custom)>h5:first-child+pre,.theme-default-content:not(.custom)>h6:first-child+.custom-block,.theme-default-content:not(.custom)>h6:first-child+p,.theme-default-content:not(.custom)>h6:first-child+pre{margin-top:2rem}h1:focus .header-anchor,h1:hover .header-anchor,h2:focus .header-anchor,h2:hover .header-anchor,h3:focus .header-anchor,h3:hover .header-anchor,h4:focus .header-anchor,h4:hover .header-anchor,h5:focus .header-anchor,h5:hover .header-anchor,h6:focus .header-anchor,h6:hover .header-anchor{opacity:1}h1{font-size:2.2rem}h2{font-size:1.65rem;padding-bottom:.3rem;border-bottom:1px solid #eaecef}h3{font-size:1.35rem}a.header-anchor{font-size:.85em;float:left;margin-left:-.87em;padding-right:.23em;margin-top:.125em;opacity:0}a.header-anchor:focus,a.header-anchor:hover{text-decoration:none}.line-number,code,kbd{font-family:source-code-pro,Menlo,Monaco,Consolas,Courier New,monospace}ol,p,ul{line-height:1.7}hr{border:0;border-top:1px solid #eaecef}table{border-collapse:collapse;margin:1rem 0;display:block;overflow-x:auto}tr{border-top:1px solid #dfe2e5}tr:nth-child(2n){background-color:#f6f8fa}td,th{border:1px solid #dfe2e5;padding:.6em 1em}.theme-container.sidebar-open .sidebar-mask{display:block}.theme-container.no-navbar .theme-default-content:not(.custom)>h1,.theme-container.no-navbar h2,.theme-container.no-navbar h3,.theme-container.no-navbar h4,.theme-container.no-navbar h5,.theme-container.no-navbar h6{margin-top:1.5rem;padding-top:0}.theme-container.no-navbar .sidebar{top:0}@media (min-width:720px){.theme-container.no-sidebar .sidebar{display:none}.theme-container.no-sidebar .page{padding-left:0}}@media (max-width:959px){.sidebar{font-size:15px;width:16.4rem}.page{padding-left:16.4rem}}@media (max-width:719px){.sidebar{top:0;padding-top:4.6rem;transform:translateX(-100%);transition:transform .2s ease}.page{padding-left:0}.theme-container.sidebar-open .sidebar{transform:translateX(0)}.theme-container.no-navbar .sidebar{padding-top:0}}@media (max-width:419px){h1{font-size:1.9rem}.theme-default-content div[class*=language-]{margin:.85rem -1.5rem;border-radius:0}}*,:after,:before{--tw-border-spacing-x:0;--tw-border-spacing-y:0;--tw-translate-x:0;--tw-translate-y:0;--tw-rotate:0;--tw-skew-x:0;--tw-skew-y:0;--tw-scale-x:1;--tw-scale-y:1;--tw-scroll-snap-strictness:proximity;--tw-ring-offset-width:0px;--tw-ring-offset-color:#fff;--tw-ring-color:rgb(59 130 246/0.5);--tw-ring-offset-shadow:0 0 transparent;--tw-ring-shadow:0 0 transparent;--tw-shadow:0 0 transparent;--tw-shadow-colored:0 0 transparent}::-webkit-backdrop{--tw-border-spacing-x:0;--tw-border-spacing-y:0;--tw-translate-x:0;--tw-translate-y:0;--tw-rotate:0;--tw-skew-x:0;--tw-skew-y:0;--tw-scale-x:1;--tw-scale-y:1;--tw-scroll-snap-strictness:proximity;--tw-ring-offset-width:0px;--tw-ring-offset-color:#fff;--tw-ring-color:rgb(59 130 246/0.5);--tw-ring-offset-shadow:0 0 transparent;--tw-ring-shadow:0 0 transparent;--tw-shadow:0 0 transparent;--tw-shadow-colored:0 0 transparent}::backdrop{--tw-border-spacing-x:0;--tw-border-spacing-y:0;--tw-translate-x:0;--tw-translate-y:0;--tw-rotate:0;--tw-skew-x:0;--tw-skew-y:0;--tw-scale-x:1;--tw-scale-y:1;--tw-scroll-snap-strictness:proximity;--tw-ring-offset-width:0px;--tw-ring-offset-color:#fff;--tw-ring-color:rgb(59 130 246/0.5);--tw-ring-offset-shadow:0 0 transparent;--tw-ring-shadow:0 0 transparent;--tw-shadow:0 0 transparent;--tw-shadow-colored:0 0 transparent}body{font-family:Lato,sans-serif;max-width:1920px;margin:0 auto;color:#000}.navbar{border-bottom:1px solid #f8f8f8;color:#000}.navbar .site-name{display:none}.dropdown-wrapper .dropdown-title{pointer-events:none}@media (max-width:720px){.dropdown-wrapper .dropdown-title{pointer-events:auto;outline:none}}.dropdown-wrapper span{color:#000}.theme-default-content:not(.custom){max-width:992px!important}img.logo{padding-left:10px}h1,h2,h3,p>span{font-weight:500;color:#000}.custom-block.tip{background-color:#f3f5f7}#nprogress{pointer-events:none}#nprogress .bar{background:#1a98ff;position:fixed;z-index:1031;top:0;left:0;width:100%;height:2px}#nprogress .peg{display:block;position:absolute;right:0;width:100px;height:100%;box-shadow:0 0 10px #1a98ff,0 0 5px #1a98ff;opacity:1;transform:rotate(3deg) translateY(-4px)}#nprogress .spinner{display:block;position:fixed;z-index:1031;top:15px;right:15px}#nprogress .spinner-icon{width:18px;height:18px;box-sizing:border-box;border-color:#1a98ff transparent transparent #1a98ff;border-style:solid;border-width:2px;border-radius:50%;-webkit-animation:nprogress-spinner .4s linear infinite;animation:nprogress-spinner .4s linear infinite}.nprogress-custom-parent{overflow:hidden;position:relative}.nprogress-custom-parent #nprogress .bar,.nprogress-custom-parent #nprogress .spinner{position:absolute}@-webkit-keyframes nprogress-spinner{0%{transform:rotate(0deg)}to{transform:rotate(1turn)}}@keyframes nprogress-spinner{0%{transform:rotate(0deg)}to{transform:rotate(1turn)}}html{scroll-behavior:smooth}.go-to-top[data-v-5fd4ef0c]{cursor:pointer;position:fixed;bottom:2rem;right:2.5rem;width:2rem;color:#1a98ff;z-index:1}.go-to-top[data-v-5fd4ef0c]:hover{color:#5fb7ff}@media (max-width:959px){.go-to-top[data-v-5fd4ef0c]{display:none}}.fade-enter-active[data-v-5fd4ef0c],.fade-leave-active[data-v-5fd4ef0c]{transition:opacity .3s}.fade-enter[data-v-5fd4ef0c],.fade-leave-to[data-v-5fd4ef0c]{opacity:0}.icon.outbound{color:#aaa;display:inline-block;vertical-align:middle;position:relative;top:-1px}.sr-only{position:absolute;width:1px;height:1px;padding:0;margin:-1px;overflow:hidden;clip:rect(0,0,0,0);white-space:nowrap;border-width:0}.home{padding:4.6rem 2rem 0;max-width:960px;margin:0 auto;display:block}.home .hero{text-align:center}.home .hero img{max-width:100%;max-height:280px;display:block;margin:3rem auto 1.5rem}.home .hero h1{font-size:3rem}.home .hero .action,.home .hero .description,.home .hero h1{margin:1.8rem auto}.home .hero .description{max-width:35rem;font-size:1.6rem;line-height:1.3;color:#6a8bad}.home .hero .action-button{display:inline-block;font-size:1.2rem;color:#fff;background-color:#1a98ff;padding:.8rem 1.6rem;border-radius:4px;transition:background-color .1s ease;box-sizing:border-box;border-bottom:1px solid #008bfd}.home .hero .action-button:hover{background-color:#31a2ff}.home .features{border-top:1px solid #eaecef;padding:1.2rem 0;margin-top:2.5rem;display:flex;flex-wrap:wrap;align-items:flex-start;align-content:stretch;justify-content:space-between}.home .feature{flex-grow:1;flex-basis:30%;max-width:30%}.home .feature h2{font-size:1.4rem;font-weight:500;border-bottom:none;padding-bottom:0;color:#3a5169}.home .feature p{color:#4e6e8e}.home .footer{padding:2.5rem;border-top:1px solid #eaecef;text-align:center;color:#4e6e8e}@media (max-width:719px){.home .features{flex-direction:column}.home .feature{max-width:100%;padding:0 2.5rem}}@media (max-width:419px){.home{padding-left:1.5rem;padding-right:1.5rem}.home .hero img{max-height:210px;margin:2rem auto 1.2rem}.home .hero h1{font-size:2rem}.home .hero .action,.home .hero .description,.home .hero h1{margin:1.2rem auto}.home .hero .description{font-size:1.2rem}.home .hero .action-button{font-size:1rem;padding:.6rem 1.2rem}.home .feature h2{font-size:1.25rem}}.search-box{display:inline-block;position:relative;margin-right:1rem}.search-box input{cursor:text;width:10rem;height:2rem;color:#4e6e8e;display:inline-block;border:1px solid #cfd4db;border-radius:2rem;font-size:.9rem;line-height:2rem;padding:0 .5rem 0 2rem;outline:none;transition:all .2s ease;background:#fff url(/assets/img/search.83621669.svg) .6rem .5rem no-repeat;background-size:1rem}.search-box input:focus{cursor:auto;border-color:#1a98ff}.search-box .suggestions{background:#fff;width:20rem;position:absolute;top:2rem;border:1px solid #cfd4db;border-radius:6px;padding:.4rem;list-style-type:none}.search-box .suggestions.align-right{right:0}.search-box .suggestion{line-height:1.4;padding:.4rem .6rem;border-radius:4px;cursor:pointer}.search-box .suggestion a{white-space:normal;color:#5d82a6}.search-box .suggestion a .page-title{font-weight:600}.search-box .suggestion a .header{font-size:.9em;margin-left:.25em}.search-box .suggestion.focused{background-color:#f3f4f5}.search-box .suggestion.focused a{color:#1a98ff}@media (max-width:959px){.search-box input{cursor:pointer;width:0;border-color:transparent;position:relative}.search-box input:focus{cursor:text;left:0;width:10rem}}@media (-ms-high-contrast:none){.search-box input{height:2rem}}@media (max-width:959px) and (min-width:719px){.search-box .suggestions{left:0}}@media (max-width:719px){.search-box{margin-right:0}.search-box input{left:1rem}.search-box .suggestions{right:0}}@media (max-width:419px){.search-box .suggestions{width:calc(100vw - 4rem)}.search-box input:focus{width:8rem}}.sidebar-button{cursor:pointer;display:none;width:1.25rem;height:1.25rem;position:absolute;padding:.6rem;top:.6rem;left:1rem}.sidebar-button .icon{display:block;width:1.25rem;height:1.25rem}@media (max-width:719px){.sidebar-button{display:block}}.dropdown-enter,.dropdown-leave-to{height:0!important}.dropdown-wrapper{cursor:pointer}.dropdown-wrapper .dropdown-title,.dropdown-wrapper .mobile-dropdown-title{display:block;font-size:.9rem;font-family:inherit;cursor:inherit;padding:inherit;line-height:1.4rem;background:transparent;border:none;font-weight:500;color:#2c3e50}.dropdown-wrapper .dropdown-title:hover,.dropdown-wrapper .mobile-dropdown-title:hover{border-color:transparent}.dropdown-wrapper .dropdown-title .arrow,.dropdown-wrapper .mobile-dropdown-title .arrow{vertical-align:middle;margin-top:-1px;margin-left:.4rem}.dropdown-wrapper .mobile-dropdown-title{display:none;font-weight:600}.dropdown-wrapper .mobile-dropdown-title font-size inherit:hover{color:#1a98ff}.dropdown-wrapper .nav-dropdown .dropdown-item{color:inherit;line-height:1.7rem}.dropdown-wrapper .nav-dropdown .dropdown-item h4{margin:.45rem 0 0;border-top:1px solid #eee;padding:1rem 1.5rem .45rem 1.25rem}.dropdown-wrapper .nav-dropdown .dropdown-item .dropdown-subitem-wrapper{padding:0;list-style:none}.dropdown-wrapper .nav-dropdown .dropdown-item .dropdown-subitem-wrapper .dropdown-subitem{font-size:.9em}.dropdown-wrapper .nav-dropdown .dropdown-item a{display:block;line-height:1.7rem;position:relative;border-bottom:none;font-weight:400;margin-bottom:0;padding:0 1.5rem 0 1.25rem}.dropdown-wrapper .nav-dropdown .dropdown-item a.router-link-active,.dropdown-wrapper .nav-dropdown .dropdown-item a:hover{color:#1a98ff}.dropdown-wrapper .nav-dropdown .dropdown-item a.router-link-active:after{content:"";width:0;height:0;border-left:5px solid #1a98ff;border-top:3px solid transparent;border-bottom:3px solid transparent;position:absolute;top:calc(50% - 2px);left:9px}.dropdown-wrapper .nav-dropdown .dropdown-item:first-child h4{margin-top:0;padding-top:0;border-top:0}@media (max-width:719px){.dropdown-wrapper.open .dropdown-title{margin-bottom:.5rem}.dropdown-wrapper .dropdown-title{display:none}.dropdown-wrapper .mobile-dropdown-title{display:block}.dropdown-wrapper .nav-dropdown{transition:height .1s ease-out;overflow:hidden}.dropdown-wrapper .nav-dropdown .dropdown-item h4{border-top:0;margin-top:0;padding-top:0}.dropdown-wrapper .nav-dropdown .dropdown-item>a,.dropdown-wrapper .nav-dropdown .dropdown-item h4{font-size:15px;line-height:2rem}.dropdown-wrapper .nav-dropdown .dropdown-item .dropdown-subitem{font-size:14px;padding-left:1rem}}@media (min-width:719px){.dropdown-wrapper{height:1.8rem}.dropdown-wrapper.open .nav-dropdown,.dropdown-wrapper:hover .nav-dropdown{display:block!important}.dropdown-wrapper.open:blur{display:none}.dropdown-wrapper .nav-dropdown{display:none;height:auto!important;box-sizing:border-box;max-height:calc(100vh - 2.7rem);overflow-y:auto;position:absolute;top:100%;right:0;background-color:#fff;padding:.6rem 0;border:1px solid;border-color:#ddd #ddd #ccc;text-align:left;border-radius:.25rem;white-space:nowrap;margin:0}}.nav-links{display:inline-block}.nav-links a{line-height:1.4rem;color:inherit}.nav-links a.router-link-active,.nav-links a:hover{color:#1a98ff}.nav-links .nav-item{position:relative;display:inline-block;margin-left:1.5rem;line-height:2rem}.nav-links .nav-item:first-child{margin-left:0}.nav-links .repo-link{margin-left:1.5rem}@media (max-width:719px){.nav-links .nav-item,.nav-links .repo-link{margin-left:0}}@media (min-width:719px){.nav-links a.router-link-active,.nav-links a:hover{color:#2c3e50}.nav-item>a:not(.external).router-link-active,.nav-item>a:not(.external):hover{margin-bottom:-2px;border-bottom:2px solid #2ca0ff}}.navbaricon{filter:invert(10%) sepia(66%) saturate(12%) hue-rotate(28deg) brightness(91%) contrast(93%)}.navbaricon:hover{filter:invert(55%) sepia(9%) saturate(3922%) hue-rotate(329deg) brightness(97%) contrast(85%)}.navbar{padding:.7rem 1.5rem;line-height:3.2rem}.navbar a,.navbar img,.navbar span{display:inline-block}.navbar .logo{height:3.2rem;min-width:3.2rem;margin-right:.8rem;vertical-align:top;position:relative;z-index:10}.navbar .site-name{font-size:1.3rem;font-weight:600;color:#2c3e50;position:relative}.navbar .links{padding-left:13.5rem;box-sizing:border-box;background-color:#fff;white-space:nowrap;font-size:.9rem;position:absolute;right:1.5rem;top:.7rem;display:flex}.navbar .links .external{opacity:.5}.navbar .links .search-box{flex:0 0 auto;vertical-align:top}@media (max-width:719px){.navbar{padding-left:4rem}.navbar .can-hide{display:none}.navbar .links{padding-left:1.5rem}.navbar .site-name{width:calc(100vw - 9.4rem);overflow:hidden;white-space:nowrap;text-overflow:ellipsis}}.page-edit{max-width:740px;margin:0 auto;padding:2rem 2.5rem}@media (max-width:959px){.page-edit{padding:2rem}}@media (max-width:419px){.page-edit{padding:1.5rem}}.page-edit{padding-top:1rem;padding-bottom:1rem;overflow:auto}.page-edit .edit-link{display:inline-block}.page-edit .edit-link a{color:#4e6e8e;margin-right:.25rem}.page-edit .last-updated{float:right;font-size:.9em}.page-edit .last-updated .prefix{font-weight:500;color:#4e6e8e}.page-edit .last-updated .time{font-weight:400;color:#767676}@media (max-width:719px){.page-edit .edit-link{margin-bottom:.5rem}.page-edit .last-updated{font-size:.8em;float:none;text-align:left}}.page-nav{max-width:740px;margin:0 auto;padding:2rem 2.5rem}@media (max-width:959px){.page-nav{padding:2rem}}@media (max-width:419px){.page-nav{padding:1.5rem}}.page-nav{padding-top:1rem;padding-bottom:0}.page-nav .inner{min-height:2rem;margin-top:0;border-top:1px solid #eaecef;padding-top:1rem;overflow:auto}.page-nav .next{float:right}.page{padding-bottom:2rem;display:block}.sidebar-group .sidebar-group{padding-left:.5em}.sidebar-group:not(.collapsable) .sidebar-heading:not(.clickable){cursor:auto;color:inherit}.sidebar-group.is-sub-group{padding-left:0}.sidebar-group.is-sub-group>.sidebar-heading{font-size:.95em;line-height:1.4;font-weight:400;padding-left:2rem}.sidebar-group.is-sub-group>.sidebar-heading:not(.clickable){opacity:.5}.sidebar-group.is-sub-group>.sidebar-group-items{padding-left:1rem}.sidebar-group.is-sub-group>.sidebar-group-items>li>.sidebar-link{font-size:.95em;border-left:none}.sidebar-group.depth-2>.sidebar-heading{border-left:none}.sidebar-heading{color:#2c3e50;transition:color .15s ease;cursor:pointer;font-size:1.1em;font-weight:700;padding:.35rem 1.5rem .35rem 1.25rem;width:100%;box-sizing:border-box;margin:0;border-left:.25rem solid transparent}.sidebar-heading.open,.sidebar-heading:hover{color:inherit}.sidebar-heading .arrow{position:relative;top:-.12em;left:.5em}.sidebar-heading.clickable.active{font-weight:600;color:#1a98ff;border-left-color:#1a98ff}.sidebar-heading.clickable:hover{color:#1a98ff}.sidebar-group-items{transition:height .1s ease-out;font-size:.95em;overflow:hidden}.sidebar .sidebar-sub-headers{padding-left:1rem;font-size:.95em}a.sidebar-link{font-size:1em;font-weight:400;display:inline-block;color:#2c3e50;border-left:.25rem solid transparent;padding:.35rem 1rem .35rem 1.25rem;line-height:1.4;width:100%;box-sizing:border-box}a.sidebar-link:hover{color:#1a98ff}a.sidebar-link.active{font-weight:600;color:#1a98ff;border-left-color:#1a98ff}.sidebar-group a.sidebar-link{padding-left:2rem}.sidebar-sub-headers a.sidebar-link{padding-top:.25rem;padding-bottom:.25rem;border-left:none}.sidebar-sub-headers a.sidebar-link.active{font-weight:500}.sidebar ul{padding:0;margin:0;list-style-type:none}.sidebar a{display:inline-block}.sidebar .nav-links{display:none;border-bottom:1px solid #eaecef;padding:.5rem 0 .75rem}.sidebar .nav-links a{font-weight:600}.sidebar .nav-links .nav-item,.sidebar .nav-links .repo-link{display:block;line-height:1.25rem;font-size:1.1em;padding:.5rem 0 .5rem 1.5rem}.sidebar>.sidebar-links{padding:1.5rem 0}.sidebar>.sidebar-links>li>a.sidebar-link{font-size:1.1em;line-height:1.7;font-weight:700}.sidebar>.sidebar-links>li:not(:first-child){margin-top:.75rem}@media (max-width:719px){.sidebar .nav-links{display:block}.sidebar .nav-links .dropdown-wrapper .nav-dropdown .dropdown-item a.router-link-active:after{top:calc(1rem - 2px)}.sidebar>.sidebar-links{padding:1rem 0}}.highlighter td{empty-cells:show}.highlighter tr{background-color:#fff}.highlighter .add{background-color:#7fff7f}.highlighter .remove{background-color:#ff7f7f}.highlighter td.modify{background-color:#7f7fff}.highlighter td.conflict{background-color:red}.highlighter .spec{background-color:#aaa}.highlighter .move{background-color:#ffa}.highlighter .null{color:#888}.highlighter table{border-collapse:collapse}.highlighter td,.highlighter th{border:1px solid #2d4068;padding:3px 7px 2px}.highlighter .header,.highlighter th{background-color:#aaf;font-weight:700;padding-bottom:4px;padding-top:5px;text-align:left}.highlighter tr:first-child td{border-top:1px solid #2d4068;background-color:#fff}.highlighter td:first-child{border-left:1px solid #2d4068}.highlighter table{margin:0 auto 20px}.highlighter tr{border-top:none}.highlighter tr.desc td{border:none;font-style:italic;text-align:center}.highlighter td.desc{font-style:italic}.highlighter td.desc,.highlighter tr.blank td{border:none;background-color:#fff}code[class*=language-],pre[class*=language-]{color:#ccc;background:none;font-family:Consolas,Monaco,Andale Mono,Ubuntu Mono,monospace;font-size:1em;text-align:left;white-space:pre;word-spacing:normal;word-break:normal;word-wrap:normal;line-height:1.5;-moz-tab-size:4;-o-tab-size:4;tab-size:4;-webkit-hyphens:none;-moz-hyphens:none;-ms-hyphens:none;hyphens:none}pre[class*=language-]{padding:1em;margin:.5em 0;overflow:auto}:not(pre)>code[class*=language-],pre[class*=language-]{background:#2d2d2d}:not(pre)>code[class*=language-]{padding:.1em;border-radius:.3em;white-space:normal}.token.block-comment,.token.cdata,.token.comment,.token.doctype,.token.prolog{color:#999}.token.punctuation{color:#ccc}.token.attr-name,.token.deleted,.token.namespace,.token.tag{color:#e2777a}.token.function-name{color:#6196cc}.token.boolean,.token.function,.token.number{color:#f08d49}.token.class-name,.token.constant,.token.property,.token.symbol{color:#f8c555}.token.atrule,.token.builtin,.token.important,.token.keyword,.token.selector{color:#cc99cd}.token.attr-value,.token.char,.token.regex,.token.string,.token.variable{color:#7ec699}.token.entity,.token.operator,.token.url{color:#67cdcc}.token.bold,.token.important{font-weight:700}.token.italic{font-style:italic}.token.entity{cursor:help}.token.inserted{color:green}div[class*=language-]{position:relative;background-color:#282c34;border-radius:6px}div[class*=language-] .highlight-lines{-webkit-user-select:none;-moz-user-select:none;-ms-user-select:none;user-select:none;padding-top:1.3rem;position:absolute;top:0;left:0;width:100%;line-height:1.4}div[class*=language-] .highlight-lines .highlighted{background-color:rgba(0,0,0,.66)}div[class*=language-] pre,div[class*=language-] pre[class*=language-]{background:transparent;position:relative;z-index:1}div[class*=language-]:before{position:absolute;z-index:3;top:.8em;right:1em;font-size:.75rem;color:hsla(0,0%,100%,.4)}div[class*=language-]:not(.line-numbers-mode) .line-numbers-wrapper{display:none}div[class*=language-].line-numbers-mode .highlight-lines .highlighted{position:relative}div[class*=language-].line-numbers-mode .highlight-lines .highlighted:before{content:" ";position:absolute;z-index:3;left:0;top:0;display:block;width:3.5rem;height:100%;background-color:rgba(0,0,0,.66)}div[class*=language-].line-numbers-mode pre{padding-left:4.5rem;vertical-align:middle}div[class*=language-].line-numbers-mode .line-numbers-wrapper{position:absolute;top:0;width:3.5rem;text-align:center;color:hsla(0,0%,100%,.3);padding:1.25rem 0;line-height:1.4}div[class*=language-].line-numbers-mode .line-numbers-wrapper .line-number,div[class*=language-].line-numbers-mode .line-numbers-wrapper br{-webkit-user-select:none;-moz-user-select:none;-ms-user-select:none;user-select:none}div[class*=language-].line-numbers-mode .line-numbers-wrapper .line-number{position:relative;z-index:4;font-size:.85em}div[class*=language-].line-numbers-mode:after{content:"";position:absolute;z-index:2;top:0;left:0;width:3.5rem;height:100%;border-radius:6px 0 0 6px;border-right:1px solid rgba(0,0,0,.66);background-color:#282c34}div[class~=language-js]:before{content:"js"}div[class~=language-ts]:before{content:"ts"}div[class~=language-html]:before{content:"html"}div[class~=language-md]:before{content:"md"}div[class~=language-vue]:before{content:"vue"}div[class~=language-css]:before{content:"css"}div[class~=language-sass]:before{content:"sass"}div[class~=language-scss]:before{content:"scss"}div[class~=language-less]:before{content:"less"}div[class~=language-stylus]:before{content:"stylus"}div[class~=language-go]:before{content:"go"}div[class~=language-java]:before{content:"java"}div[class~=language-c]:before{content:"c"}div[class~=language-sh]:before{content:"sh"}div[class~=language-yaml]:before{content:"yaml"}div[class~=language-py]:before{content:"py"}div[class~=language-docker]:before{content:"docker"}div[class~=language-dockerfile]:before{content:"dockerfile"}div[class~=language-makefile]:before{content:"makefile"}div[class~=language-javascript]:before{content:"js"}div[class~=language-typescript]:before{content:"ts"}div[class~=language-markup]:before{content:"html"}div[class~=language-markdown]:before{content:"md"}div[class~=language-json]:before{content:"json"}div[class~=language-ruby]:before{content:"rb"}div[class~=language-python]:before{content:"py"}div[class~=language-bash]:before{content:"sh"}div[class~=language-php]:before{content:"php"}div.language-mermaid{background-color:inherit}.theme-code-block[data-v-759a7d02]{display:none}.theme-code-block__active[data-v-759a7d02]{display:block}.theme-code-block>pre[data-v-759a7d02]{background-color:orange}.badge[data-v-15b7b770]{display:inline-block;font-size:14px;height:18px;line-height:18px;border-radius:3px;padding:0 6px;color:#fff}.badge.green[data-v-15b7b770],.badge.tip[data-v-15b7b770],.badge[data-v-15b7b770]{background-color:#42b983}.badge.error[data-v-15b7b770]{background-color:#da5961}.badge.warn[data-v-15b7b770],.badge.warning[data-v-15b7b770],.badge.yellow[data-v-15b7b770]{background-color:#e7c000}.badge+.badge[data-v-15b7b770]{margin-left:5px}.theme-code-group__nav[data-v-deefee04]{margin-bottom:-35px;background-color:#282c34;padding-bottom:22px;border-top-left-radius:6px;border-top-right-radius:6px;padding-left:10px;padding-top:10px}.theme-code-group__ul[data-v-deefee04]{margin:auto 0;padding-left:0;display:inline-flex;list-style:none}.theme-code-group__nav-tab[data-v-deefee04]{border:0;padding:5px;cursor:pointer;background-color:transparent;font-size:.85em;line-height:1.4;color:hsla(0,0%,100%,.9);font-weight:600}.theme-code-group__nav-tab-active[data-v-deefee04]{border-bottom:1px solid #42b983}.pre-blank[data-v-deefee04]{color:#42b983} \ No newline at end of file +.theme-default-content code{color:#476582;padding:.25rem .5rem;margin:0;font-size:.85em;background-color:rgba(27,31,35,.05);border-radius:3px}.theme-default-content code .token.deleted{color:#ec5975}.theme-default-content code .token.inserted{color:#1a98ff}.theme-default-content pre,.theme-default-content pre[class*=language-]{line-height:1.4;padding:1.25rem 1.5rem;margin:.85rem 0;background-color:#282c34;border-radius:6px;overflow:auto}.theme-default-content pre[class*=language-] code,.theme-default-content pre code{color:#fff;padding:0;background-color:transparent;border-radius:0}.custom-block .custom-block-title{font-weight:600;margin-bottom:-.4rem}.custom-block.danger,.custom-block.tip,.custom-block.warning{padding:.1rem 1.5rem;border-left-width:.5rem;border-left-style:solid;margin:1rem 0}.custom-block.tip{border-color:#42b983}.custom-block.warning{background-color:rgba(255,229,100,.3);border-color:#e7c000;color:#6b5900}.custom-block.warning .custom-block-title{color:#b29400}.custom-block.warning a{color:#2c3e50}.custom-block.danger{background-color:#ffe6e6;border-color:#c00;color:#4d0000}.custom-block.danger .custom-block-title{color:#900}.custom-block.danger a{color:#2c3e50}.custom-block.details{display:block;position:relative;border-radius:2px;margin:1.6em 0;padding:1.6em;background-color:#eee}.custom-block.details h4{margin-top:0}.custom-block.details figure:last-child,.custom-block.details p:last-child{margin-bottom:0;padding-bottom:0}.custom-block.details summary{outline:none;cursor:pointer}.arrow{display:inline-block;width:0;height:0}.arrow.up{border-bottom:6px solid #ccc}.arrow.down,.arrow.up{border-left:4px solid transparent;border-right:4px solid transparent}.arrow.down{border-top:6px solid #ccc}.arrow.right{border-left:6px solid #ccc}.arrow.left,.arrow.right{border-top:4px solid transparent;border-bottom:4px solid transparent}.arrow.left{border-right:6px solid #ccc}.theme-default-content:not(.custom){max-width:740px;margin:0 auto;padding:2rem 2.5rem}@media (max-width:959px){.theme-default-content:not(.custom){padding:2rem}}@media (max-width:419px){.theme-default-content:not(.custom){padding:1.5rem}}.table-of-contents .badge{vertical-align:middle}body,html{padding:0;margin:0;background-color:#fff}body{font-family:-apple-system,BlinkMacSystemFont,Segoe UI,Roboto,Oxygen,Ubuntu,Cantarell,Fira Sans,Droid Sans,Helvetica Neue,sans-serif;-webkit-font-smoothing:antialiased;-moz-osx-font-smoothing:grayscale;font-size:16px;color:#2c3e50}.page{padding-left:20rem}.navbar{z-index:20;right:0;height:4.6rem;background-color:#fff;box-sizing:border-box;border-bottom:1px solid #eaecef}.navbar,.sidebar-mask{position:fixed;top:0;left:0}.sidebar-mask{z-index:9;width:100vw;height:100vh;display:none}.sidebar{font-size:16px;background-color:#fff;width:20rem;position:fixed;z-index:10;margin:0;top:4.6rem;left:0;bottom:0;box-sizing:border-box;border-right:1px solid #eaecef;overflow-y:auto}.theme-default-content:not(.custom)>:first-child{margin-top:4.6rem}.theme-default-content:not(.custom) a:hover{text-decoration:underline}.theme-default-content:not(.custom) p.demo{padding:1rem 1.5rem;border:1px solid #ddd;border-radius:4px}.theme-default-content:not(.custom) img{max-width:100%}.theme-default-content.custom{padding:0;margin:0}.theme-default-content.custom img{max-width:100%}a{font-weight:500;text-decoration:none}a,p a code{color:#1a98ff}p a code{font-weight:400}kbd{background:#eee;border:.15rem solid #ddd;border-bottom:.25rem solid #ddd;border-radius:.15rem;padding:0 .15em}blockquote{font-size:1rem;color:#999;border-left:.2rem solid #dfe2e5;margin:1rem 0;padding:.25rem 0 .25rem 1rem}blockquote>p{margin:0}ol,ul{padding-left:1.2em}strong{font-weight:600}h1,h2,h3,h4,h5,h6{font-weight:600;line-height:1.25}.theme-default-content:not(.custom)>h1,.theme-default-content:not(.custom)>h2,.theme-default-content:not(.custom)>h3,.theme-default-content:not(.custom)>h4,.theme-default-content:not(.custom)>h5,.theme-default-content:not(.custom)>h6{margin-top:-4.1rem;padding-top:5.6rem;margin-bottom:0}.theme-default-content:not(.custom)>h1:first-child,.theme-default-content:not(.custom)>h2:first-child,.theme-default-content:not(.custom)>h3:first-child,.theme-default-content:not(.custom)>h4:first-child,.theme-default-content:not(.custom)>h5:first-child,.theme-default-content:not(.custom)>h6:first-child{margin-top:-1.5rem;margin-bottom:1rem}.theme-default-content:not(.custom)>h1:first-child+.custom-block,.theme-default-content:not(.custom)>h1:first-child+p,.theme-default-content:not(.custom)>h1:first-child+pre,.theme-default-content:not(.custom)>h2:first-child+.custom-block,.theme-default-content:not(.custom)>h2:first-child+p,.theme-default-content:not(.custom)>h2:first-child+pre,.theme-default-content:not(.custom)>h3:first-child+.custom-block,.theme-default-content:not(.custom)>h3:first-child+p,.theme-default-content:not(.custom)>h3:first-child+pre,.theme-default-content:not(.custom)>h4:first-child+.custom-block,.theme-default-content:not(.custom)>h4:first-child+p,.theme-default-content:not(.custom)>h4:first-child+pre,.theme-default-content:not(.custom)>h5:first-child+.custom-block,.theme-default-content:not(.custom)>h5:first-child+p,.theme-default-content:not(.custom)>h5:first-child+pre,.theme-default-content:not(.custom)>h6:first-child+.custom-block,.theme-default-content:not(.custom)>h6:first-child+p,.theme-default-content:not(.custom)>h6:first-child+pre{margin-top:2rem}h1:focus .header-anchor,h1:hover .header-anchor,h2:focus .header-anchor,h2:hover .header-anchor,h3:focus .header-anchor,h3:hover .header-anchor,h4:focus .header-anchor,h4:hover .header-anchor,h5:focus .header-anchor,h5:hover .header-anchor,h6:focus .header-anchor,h6:hover .header-anchor{opacity:1}h1{font-size:2.2rem}h2{font-size:1.65rem;padding-bottom:.3rem;border-bottom:1px solid #eaecef}h3{font-size:1.35rem}a.header-anchor{font-size:.85em;float:left;margin-left:-.87em;padding-right:.23em;margin-top:.125em;opacity:0}a.header-anchor:focus,a.header-anchor:hover{text-decoration:none}.line-number,code,kbd{font-family:source-code-pro,Menlo,Monaco,Consolas,Courier New,monospace}ol,p,ul{line-height:1.7}hr{border:0;border-top:1px solid #eaecef}table{border-collapse:collapse;margin:1rem 0;display:block;overflow-x:auto}tr{border-top:1px solid #dfe2e5}tr:nth-child(2n){background-color:#f6f8fa}td,th{border:1px solid #dfe2e5;padding:.6em 1em}.theme-container.sidebar-open .sidebar-mask{display:block}.theme-container.no-navbar .theme-default-content:not(.custom)>h1,.theme-container.no-navbar h2,.theme-container.no-navbar h3,.theme-container.no-navbar h4,.theme-container.no-navbar h5,.theme-container.no-navbar h6{margin-top:1.5rem;padding-top:0}.theme-container.no-navbar .sidebar{top:0}@media (min-width:720px){.theme-container.no-sidebar .sidebar{display:none}.theme-container.no-sidebar .page{padding-left:0}}@media (max-width:959px){.sidebar{font-size:15px;width:16.4rem}.page{padding-left:16.4rem}}@media (max-width:719px){.sidebar{top:0;padding-top:4.6rem;transform:translateX(-100%);transition:transform .2s ease}.page{padding-left:0}.theme-container.sidebar-open .sidebar{transform:translateX(0)}.theme-container.no-navbar .sidebar{padding-top:0}}@media (max-width:419px){h1{font-size:1.9rem}.theme-default-content div[class*=language-]{margin:.85rem -1.5rem;border-radius:0}}*,:after,:before{--tw-border-spacing-x:0;--tw-border-spacing-y:0;--tw-translate-x:0;--tw-translate-y:0;--tw-rotate:0;--tw-skew-x:0;--tw-skew-y:0;--tw-scale-x:1;--tw-scale-y:1;--tw-scroll-snap-strictness:proximity;--tw-ring-offset-width:0px;--tw-ring-offset-color:#fff;--tw-ring-color:rgb(59 130 246/0.5);--tw-ring-offset-shadow:0 0 transparent;--tw-ring-shadow:0 0 transparent;--tw-shadow:0 0 transparent;--tw-shadow-colored:0 0 transparent}::-webkit-backdrop{--tw-border-spacing-x:0;--tw-border-spacing-y:0;--tw-translate-x:0;--tw-translate-y:0;--tw-rotate:0;--tw-skew-x:0;--tw-skew-y:0;--tw-scale-x:1;--tw-scale-y:1;--tw-scroll-snap-strictness:proximity;--tw-ring-offset-width:0px;--tw-ring-offset-color:#fff;--tw-ring-color:rgb(59 130 246/0.5);--tw-ring-offset-shadow:0 0 transparent;--tw-ring-shadow:0 0 transparent;--tw-shadow:0 0 transparent;--tw-shadow-colored:0 0 transparent}::backdrop{--tw-border-spacing-x:0;--tw-border-spacing-y:0;--tw-translate-x:0;--tw-translate-y:0;--tw-rotate:0;--tw-skew-x:0;--tw-skew-y:0;--tw-scale-x:1;--tw-scale-y:1;--tw-scroll-snap-strictness:proximity;--tw-ring-offset-width:0px;--tw-ring-offset-color:#fff;--tw-ring-color:rgb(59 130 246/0.5);--tw-ring-offset-shadow:0 0 transparent;--tw-ring-shadow:0 0 transparent;--tw-shadow:0 0 transparent;--tw-shadow-colored:0 0 transparent}body{font-family:Lato,sans-serif;max-width:1920px;margin:0 auto;color:#000}.navbar{border-bottom:1px solid #f8f8f8;color:#000}.navbar .site-name{display:none}.dropdown-wrapper .dropdown-title{pointer-events:none}@media (max-width:720px){.dropdown-wrapper .dropdown-title{pointer-events:auto;outline:none}}.dropdown-wrapper span{color:#000}.theme-default-content:not(.custom){max-width:992px!important}img.logo{padding-left:10px}h1,h2,h3,p>span{font-weight:500;color:#000}.custom-block.tip{background-color:#f3f5f7}#nprogress{pointer-events:none}#nprogress .bar{background:#1a98ff;position:fixed;z-index:1031;top:0;left:0;width:100%;height:2px}#nprogress .peg{display:block;position:absolute;right:0;width:100px;height:100%;box-shadow:0 0 10px #1a98ff,0 0 5px #1a98ff;opacity:1;transform:rotate(3deg) translateY(-4px)}#nprogress .spinner{display:block;position:fixed;z-index:1031;top:15px;right:15px}#nprogress .spinner-icon{width:18px;height:18px;box-sizing:border-box;border-color:#1a98ff transparent transparent #1a98ff;border-style:solid;border-width:2px;border-radius:50%;-webkit-animation:nprogress-spinner .4s linear infinite;animation:nprogress-spinner .4s linear infinite}.nprogress-custom-parent{overflow:hidden;position:relative}.nprogress-custom-parent #nprogress .bar,.nprogress-custom-parent #nprogress .spinner{position:absolute}@-webkit-keyframes nprogress-spinner{0%{transform:rotate(0deg)}to{transform:rotate(1turn)}}@keyframes nprogress-spinner{0%{transform:rotate(0deg)}to{transform:rotate(1turn)}}html{scroll-behavior:smooth}.go-to-top[data-v-5fd4ef0c]{cursor:pointer;position:fixed;bottom:2rem;right:2.5rem;width:2rem;color:#1a98ff;z-index:1}.go-to-top[data-v-5fd4ef0c]:hover{color:#5fb7ff}@media (max-width:959px){.go-to-top[data-v-5fd4ef0c]{display:none}}.fade-enter-active[data-v-5fd4ef0c],.fade-leave-active[data-v-5fd4ef0c]{transition:opacity .3s}.fade-enter[data-v-5fd4ef0c],.fade-leave-to[data-v-5fd4ef0c]{opacity:0}.icon.outbound{color:#aaa;display:inline-block;vertical-align:middle;position:relative;top:-1px}.sr-only{position:absolute;width:1px;height:1px;padding:0;margin:-1px;overflow:hidden;clip:rect(0,0,0,0);white-space:nowrap;border-width:0}.home{padding:4.6rem 2rem 0;max-width:960px;margin:0 auto;display:block}.home .hero{text-align:center}.home .hero img{max-width:100%;max-height:280px;display:block;margin:3rem auto 1.5rem}.home .hero h1{font-size:3rem}.home .hero .action,.home .hero .description,.home .hero h1{margin:1.8rem auto}.home .hero .description{max-width:35rem;font-size:1.6rem;line-height:1.3;color:#6a8bad}.home .hero .action-button{display:inline-block;font-size:1.2rem;color:#fff;background-color:#1a98ff;padding:.8rem 1.6rem;border-radius:4px;transition:background-color .1s ease;box-sizing:border-box;border-bottom:1px solid #008bfd}.home .hero .action-button:hover{background-color:#31a2ff}.home .features{border-top:1px solid #eaecef;padding:1.2rem 0;margin-top:2.5rem;display:flex;flex-wrap:wrap;align-items:flex-start;align-content:stretch;justify-content:space-between}.home .feature{flex-grow:1;flex-basis:30%;max-width:30%}.home .feature h2{font-size:1.4rem;font-weight:500;border-bottom:none;padding-bottom:0;color:#3a5169}.home .feature p{color:#4e6e8e}.home .footer{padding:2.5rem;border-top:1px solid #eaecef;text-align:center;color:#4e6e8e}@media (max-width:719px){.home .features{flex-direction:column}.home .feature{max-width:100%;padding:0 2.5rem}}@media (max-width:419px){.home{padding-left:1.5rem;padding-right:1.5rem}.home .hero img{max-height:210px;margin:2rem auto 1.2rem}.home .hero h1{font-size:2rem}.home .hero .action,.home .hero .description,.home .hero h1{margin:1.2rem auto}.home .hero .description{font-size:1.2rem}.home .hero .action-button{font-size:1rem;padding:.6rem 1.2rem}.home .feature h2{font-size:1.25rem}}.search-box{display:inline-block;position:relative;margin-right:1rem}.search-box input{cursor:text;width:10rem;height:2rem;color:#4e6e8e;display:inline-block;border:1px solid #cfd4db;border-radius:2rem;font-size:.9rem;line-height:2rem;padding:0 .5rem 0 2rem;outline:none;transition:all .2s ease;background:#fff url(/assets/img/search.83621669.svg) .6rem .5rem no-repeat;background-size:1rem}.search-box input:focus{cursor:auto;border-color:#1a98ff}.search-box .suggestions{background:#fff;width:20rem;position:absolute;top:2rem;border:1px solid #cfd4db;border-radius:6px;padding:.4rem;list-style-type:none}.search-box .suggestions.align-right{right:0}.search-box .suggestion{line-height:1.4;padding:.4rem .6rem;border-radius:4px;cursor:pointer}.search-box .suggestion a{white-space:normal;color:#5d82a6}.search-box .suggestion a .page-title{font-weight:600}.search-box .suggestion a .header{font-size:.9em;margin-left:.25em}.search-box .suggestion.focused{background-color:#f3f4f5}.search-box .suggestion.focused a{color:#1a98ff}@media (max-width:959px){.search-box input{cursor:pointer;width:0;border-color:transparent;position:relative}.search-box input:focus{cursor:text;left:0;width:10rem}}@media (-ms-high-contrast:none){.search-box input{height:2rem}}@media (max-width:959px) and (min-width:719px){.search-box .suggestions{left:0}}@media (max-width:719px){.search-box{margin-right:0}.search-box input{left:1rem}.search-box .suggestions{right:0}}@media (max-width:419px){.search-box .suggestions{width:calc(100vw - 4rem)}.search-box input:focus{width:8rem}}.sidebar-button{cursor:pointer;display:none;width:1.25rem;height:1.25rem;position:absolute;padding:.6rem;top:.6rem;left:1rem}.sidebar-button .icon{display:block;width:1.25rem;height:1.25rem}@media (max-width:719px){.sidebar-button{display:block}}.dropdown-enter,.dropdown-leave-to{height:0!important}.dropdown-wrapper{cursor:pointer}.dropdown-wrapper .dropdown-title,.dropdown-wrapper .mobile-dropdown-title{display:block;font-size:.9rem;font-family:inherit;cursor:inherit;padding:inherit;line-height:1.4rem;background:transparent;border:none;font-weight:500;color:#2c3e50}.dropdown-wrapper .dropdown-title:hover,.dropdown-wrapper .mobile-dropdown-title:hover{border-color:transparent}.dropdown-wrapper .dropdown-title .arrow,.dropdown-wrapper .mobile-dropdown-title .arrow{vertical-align:middle;margin-top:-1px;margin-left:.4rem}.dropdown-wrapper .mobile-dropdown-title{display:none;font-weight:600}.dropdown-wrapper .mobile-dropdown-title font-size inherit:hover{color:#1a98ff}.dropdown-wrapper .nav-dropdown .dropdown-item{color:inherit;line-height:1.7rem}.dropdown-wrapper .nav-dropdown .dropdown-item h4{margin:.45rem 0 0;border-top:1px solid #eee;padding:1rem 1.5rem .45rem 1.25rem}.dropdown-wrapper .nav-dropdown .dropdown-item .dropdown-subitem-wrapper{padding:0;list-style:none}.dropdown-wrapper .nav-dropdown .dropdown-item .dropdown-subitem-wrapper .dropdown-subitem{font-size:.9em}.dropdown-wrapper .nav-dropdown .dropdown-item a{display:block;line-height:1.7rem;position:relative;border-bottom:none;font-weight:400;margin-bottom:0;padding:0 1.5rem 0 1.25rem}.dropdown-wrapper .nav-dropdown .dropdown-item a.router-link-active,.dropdown-wrapper .nav-dropdown .dropdown-item a:hover{color:#1a98ff}.dropdown-wrapper .nav-dropdown .dropdown-item a.router-link-active:after{content:"";width:0;height:0;border-left:5px solid #1a98ff;border-top:3px solid transparent;border-bottom:3px solid transparent;position:absolute;top:calc(50% - 2px);left:9px}.dropdown-wrapper .nav-dropdown .dropdown-item:first-child h4{margin-top:0;padding-top:0;border-top:0}@media (max-width:719px){.dropdown-wrapper.open .dropdown-title{margin-bottom:.5rem}.dropdown-wrapper .dropdown-title{display:none}.dropdown-wrapper .mobile-dropdown-title{display:block}.dropdown-wrapper .nav-dropdown{transition:height .1s ease-out;overflow:hidden}.dropdown-wrapper .nav-dropdown .dropdown-item h4{border-top:0;margin-top:0;padding-top:0}.dropdown-wrapper .nav-dropdown .dropdown-item>a,.dropdown-wrapper .nav-dropdown .dropdown-item h4{font-size:15px;line-height:2rem}.dropdown-wrapper .nav-dropdown .dropdown-item .dropdown-subitem{font-size:14px;padding-left:1rem}}@media (min-width:719px){.dropdown-wrapper{height:1.8rem}.dropdown-wrapper.open .nav-dropdown,.dropdown-wrapper:hover .nav-dropdown{display:block!important}.dropdown-wrapper.open:blur{display:none}.dropdown-wrapper .nav-dropdown{display:none;height:auto!important;box-sizing:border-box;max-height:calc(100vh - 2.7rem);overflow-y:auto;position:absolute;top:100%;right:0;background-color:#fff;padding:.6rem 0;border:1px solid;border-color:#ddd #ddd #ccc;text-align:left;border-radius:.25rem;white-space:nowrap;margin:0}}.nav-links{display:inline-block}.nav-links a{line-height:1.4rem;color:inherit}.nav-links a.router-link-active,.nav-links a:hover{color:#1a98ff}.nav-links .nav-item{position:relative;display:inline-block;margin-left:1.5rem;line-height:2rem}.nav-links .nav-item:first-child{margin-left:0}.nav-links .repo-link{margin-left:1.5rem}@media (max-width:719px){.nav-links .nav-item,.nav-links .repo-link{margin-left:0}}@media (min-width:719px){.nav-links a.router-link-active,.nav-links a:hover{color:#2c3e50}.nav-item>a:not(.external).router-link-active,.nav-item>a:not(.external):hover{margin-bottom:-2px;border-bottom:2px solid #2ca0ff}}.navbaricon{filter:invert(10%) sepia(66%) saturate(12%) hue-rotate(28deg) brightness(91%) contrast(93%)}.navbaricon:hover{filter:invert(55%) sepia(9%) saturate(3922%) hue-rotate(329deg) brightness(97%) contrast(85%)}.navbar{padding:.7rem 1.5rem;line-height:3.2rem}.navbar a,.navbar img,.navbar span{display:inline-block}.navbar .logo{height:3.2rem;min-width:3.2rem;margin-right:.8rem;vertical-align:top;position:relative;z-index:10}.navbar .site-name{font-size:1.3rem;font-weight:600;color:#2c3e50;position:relative}.navbar .links{padding-left:13.5rem;box-sizing:border-box;background-color:#fff;white-space:nowrap;font-size:.9rem;position:absolute;right:1.5rem;top:.7rem;display:flex}.navbar .links .external{opacity:.5}.navbar .links .search-box{flex:0 0 auto;vertical-align:top}@media (max-width:719px){.navbar{padding-left:4rem}.navbar .can-hide{display:none}.navbar .links{padding-left:1.5rem}.navbar .site-name{width:calc(100vw - 9.4rem);overflow:hidden;white-space:nowrap;text-overflow:ellipsis}}.page-edit{max-width:740px;margin:0 auto;padding:2rem 2.5rem}@media (max-width:959px){.page-edit{padding:2rem}}@media (max-width:419px){.page-edit{padding:1.5rem}}.page-edit{padding-top:1rem;padding-bottom:1rem;overflow:auto}.page-edit .edit-link{display:inline-block}.page-edit .edit-link a{color:#4e6e8e;margin-right:.25rem}.page-edit .last-updated{float:right;font-size:.9em}.page-edit .last-updated .prefix{font-weight:500;color:#4e6e8e}.page-edit .last-updated .time{font-weight:400;color:#767676}@media (max-width:719px){.page-edit .edit-link{margin-bottom:.5rem}.page-edit .last-updated{font-size:.8em;float:none;text-align:left}}.page-nav{max-width:740px;margin:0 auto;padding:2rem 2.5rem}@media (max-width:959px){.page-nav{padding:2rem}}@media (max-width:419px){.page-nav{padding:1.5rem}}.page-nav{padding-top:1rem;padding-bottom:0}.page-nav .inner{min-height:2rem;margin-top:0;border-top:1px solid #eaecef;padding-top:1rem;overflow:auto}.page-nav .next{float:right}.page{padding-bottom:2rem;display:block}.sidebar-group .sidebar-group{padding-left:.5em}.sidebar-group:not(.collapsable) .sidebar-heading:not(.clickable){cursor:auto;color:inherit}.sidebar-group.is-sub-group{padding-left:0}.sidebar-group.is-sub-group>.sidebar-heading{font-size:.95em;line-height:1.4;font-weight:400;padding-left:2rem}.sidebar-group.is-sub-group>.sidebar-heading:not(.clickable){opacity:.5}.sidebar-group.is-sub-group>.sidebar-group-items{padding-left:1rem}.sidebar-group.is-sub-group>.sidebar-group-items>li>.sidebar-link{font-size:.95em;border-left:none}.sidebar-group.depth-2>.sidebar-heading{border-left:none}.sidebar-heading{color:#2c3e50;transition:color .15s ease;cursor:pointer;font-size:1.1em;font-weight:700;padding:.35rem 1.5rem .35rem 1.25rem;width:100%;box-sizing:border-box;margin:0;border-left:.25rem solid transparent}.sidebar-heading.open,.sidebar-heading:hover{color:inherit}.sidebar-heading .arrow{position:relative;top:-.12em;left:.5em}.sidebar-heading.clickable.active{font-weight:600;color:#1a98ff;border-left-color:#1a98ff}.sidebar-heading.clickable:hover{color:#1a98ff}.sidebar-group-items{transition:height .1s ease-out;font-size:.95em;overflow:hidden}.sidebar .sidebar-sub-headers{padding-left:1rem;font-size:.95em}a.sidebar-link{font-size:1em;font-weight:400;display:inline-block;color:#2c3e50;border-left:.25rem solid transparent;padding:.35rem 1rem .35rem 1.25rem;line-height:1.4;width:100%;box-sizing:border-box}a.sidebar-link:hover{color:#1a98ff}a.sidebar-link.active{font-weight:600;color:#1a98ff;border-left-color:#1a98ff}.sidebar-group a.sidebar-link{padding-left:2rem}.sidebar-sub-headers a.sidebar-link{padding-top:.25rem;padding-bottom:.25rem;border-left:none}.sidebar-sub-headers a.sidebar-link.active{font-weight:500}.sidebar ul{padding:0;margin:0;list-style-type:none}.sidebar a{display:inline-block}.sidebar .nav-links{display:none;border-bottom:1px solid #eaecef;padding:.5rem 0 .75rem}.sidebar .nav-links a{font-weight:600}.sidebar .nav-links .nav-item,.sidebar .nav-links .repo-link{display:block;line-height:1.25rem;font-size:1.1em;padding:.5rem 0 .5rem 1.5rem}.sidebar>.sidebar-links{padding:1.5rem 0}.sidebar>.sidebar-links>li>a.sidebar-link{font-size:1.1em;line-height:1.7;font-weight:700}.sidebar>.sidebar-links>li:not(:first-child){margin-top:.75rem}@media (max-width:719px){.sidebar .nav-links{display:block}.sidebar .nav-links .dropdown-wrapper .nav-dropdown .dropdown-item a.router-link-active:after{top:calc(1rem - 2px)}.sidebar>.sidebar-links{padding:1rem 0}}.highlighter td{empty-cells:show}.highlighter tr{background-color:#fff}.highlighter .add{background-color:#7fff7f}.highlighter .remove{background-color:#ff7f7f}.highlighter td.modify{background-color:#7f7fff}.highlighter td.conflict{background-color:red}.highlighter .spec{background-color:#aaa}.highlighter .move{background-color:#ffa}.highlighter .null{color:#888}.highlighter table{border-collapse:collapse}.highlighter td,.highlighter th{border:1px solid #2d4068;padding:3px 7px 2px}.highlighter .header,.highlighter th{background-color:#aaf;font-weight:700;padding-bottom:4px;padding-top:5px;text-align:left}.highlighter tr:first-child td{border-top:1px solid #2d4068;background-color:#fff}.highlighter td:first-child{border-left:1px solid #2d4068}.highlighter table{margin:0 auto 20px}.highlighter tr{border-top:none}.highlighter tr.desc td{border:none;font-style:italic;text-align:center}.highlighter td.desc{font-style:italic}.highlighter td.desc,.highlighter tr.blank td{border:none;background-color:#fff}code[class*=language-],pre[class*=language-]{color:#ccc;background:none;font-family:Consolas,Monaco,Andale Mono,Ubuntu Mono,monospace;font-size:1em;text-align:left;white-space:pre;word-spacing:normal;word-break:normal;word-wrap:normal;line-height:1.5;-moz-tab-size:4;-o-tab-size:4;tab-size:4;-webkit-hyphens:none;-moz-hyphens:none;-ms-hyphens:none;hyphens:none}pre[class*=language-]{padding:1em;margin:.5em 0;overflow:auto}:not(pre)>code[class*=language-],pre[class*=language-]{background:#2d2d2d}:not(pre)>code[class*=language-]{padding:.1em;border-radius:.3em;white-space:normal}.token.block-comment,.token.cdata,.token.comment,.token.doctype,.token.prolog{color:#999}.token.punctuation{color:#ccc}.token.attr-name,.token.deleted,.token.namespace,.token.tag{color:#e2777a}.token.function-name{color:#6196cc}.token.boolean,.token.function,.token.number{color:#f08d49}.token.class-name,.token.constant,.token.property,.token.symbol{color:#f8c555}.token.atrule,.token.builtin,.token.important,.token.keyword,.token.selector{color:#cc99cd}.token.attr-value,.token.char,.token.regex,.token.string,.token.variable{color:#7ec699}.token.entity,.token.operator,.token.url{color:#67cdcc}.token.bold,.token.important{font-weight:700}.token.italic{font-style:italic}.token.entity{cursor:help}.token.inserted{color:green}div[class*=language-]{position:relative;background-color:#282c34;border-radius:6px}div[class*=language-] .highlight-lines{-webkit-user-select:none;-moz-user-select:none;-ms-user-select:none;user-select:none;padding-top:1.3rem;position:absolute;top:0;left:0;width:100%;line-height:1.4}div[class*=language-] .highlight-lines .highlighted{background-color:rgba(0,0,0,.66)}div[class*=language-] pre,div[class*=language-] pre[class*=language-]{background:transparent;position:relative;z-index:1}div[class*=language-]:before{position:absolute;z-index:3;top:.8em;right:1em;font-size:.75rem;color:hsla(0,0%,100%,.4)}div[class*=language-]:not(.line-numbers-mode) .line-numbers-wrapper{display:none}div[class*=language-].line-numbers-mode .highlight-lines .highlighted{position:relative}div[class*=language-].line-numbers-mode .highlight-lines .highlighted:before{content:" ";position:absolute;z-index:3;left:0;top:0;display:block;width:3.5rem;height:100%;background-color:rgba(0,0,0,.66)}div[class*=language-].line-numbers-mode pre{padding-left:4.5rem;vertical-align:middle}div[class*=language-].line-numbers-mode .line-numbers-wrapper{position:absolute;top:0;width:3.5rem;text-align:center;color:hsla(0,0%,100%,.3);padding:1.25rem 0;line-height:1.4}div[class*=language-].line-numbers-mode .line-numbers-wrapper .line-number,div[class*=language-].line-numbers-mode .line-numbers-wrapper br{-webkit-user-select:none;-moz-user-select:none;-ms-user-select:none;user-select:none}div[class*=language-].line-numbers-mode .line-numbers-wrapper .line-number{position:relative;z-index:4;font-size:.85em}div[class*=language-].line-numbers-mode:after{content:"";position:absolute;z-index:2;top:0;left:0;width:3.5rem;height:100%;border-radius:6px 0 0 6px;border-right:1px solid rgba(0,0,0,.66);background-color:#282c34}div[class~=language-js]:before{content:"js"}div[class~=language-ts]:before{content:"ts"}div[class~=language-html]:before{content:"html"}div[class~=language-md]:before{content:"md"}div[class~=language-vue]:before{content:"vue"}div[class~=language-css]:before{content:"css"}div[class~=language-sass]:before{content:"sass"}div[class~=language-scss]:before{content:"scss"}div[class~=language-less]:before{content:"less"}div[class~=language-stylus]:before{content:"stylus"}div[class~=language-go]:before{content:"go"}div[class~=language-java]:before{content:"java"}div[class~=language-c]:before{content:"c"}div[class~=language-sh]:before{content:"sh"}div[class~=language-yaml]:before{content:"yaml"}div[class~=language-py]:before{content:"py"}div[class~=language-docker]:before{content:"docker"}div[class~=language-dockerfile]:before{content:"dockerfile"}div[class~=language-makefile]:before{content:"makefile"}div[class~=language-javascript]:before{content:"js"}div[class~=language-typescript]:before{content:"ts"}div[class~=language-markup]:before{content:"html"}div[class~=language-markdown]:before{content:"md"}div[class~=language-json]:before{content:"json"}div[class~=language-ruby]:before{content:"rb"}div[class~=language-python]:before{content:"py"}div[class~=language-bash]:before{content:"sh"}div[class~=language-php]:before{content:"php"}div.language-mermaid{background-color:inherit}.badge[data-v-15b7b770]{display:inline-block;font-size:14px;height:18px;line-height:18px;border-radius:3px;padding:0 6px;color:#fff}.badge.green[data-v-15b7b770],.badge.tip[data-v-15b7b770],.badge[data-v-15b7b770]{background-color:#42b983}.badge.error[data-v-15b7b770]{background-color:#da5961}.badge.warn[data-v-15b7b770],.badge.warning[data-v-15b7b770],.badge.yellow[data-v-15b7b770]{background-color:#e7c000}.badge+.badge[data-v-15b7b770]{margin-left:5px}.theme-code-block[data-v-759a7d02]{display:none}.theme-code-block__active[data-v-759a7d02]{display:block}.theme-code-block>pre[data-v-759a7d02]{background-color:orange}.theme-code-group__nav[data-v-deefee04]{margin-bottom:-35px;background-color:#282c34;padding-bottom:22px;border-top-left-radius:6px;border-top-right-radius:6px;padding-left:10px;padding-top:10px}.theme-code-group__ul[data-v-deefee04]{margin:auto 0;padding-left:0;display:inline-flex;list-style:none}.theme-code-group__nav-tab[data-v-deefee04]{border:0;padding:5px;cursor:pointer;background-color:transparent;font-size:.85em;line-height:1.4;color:hsla(0,0%,100%,.9);font-weight:600}.theme-code-group__nav-tab-active[data-v-deefee04]{border-bottom:1px solid #42b983}.pre-blank[data-v-deefee04]{color:#42b983} \ No newline at end of file diff --git a/assets/js/15.7a445b11.js b/assets/js/15.74a14259.js similarity index 99% rename from assets/js/15.7a445b11.js rename to assets/js/15.74a14259.js index a6abd580..c6eaa4d9 100644 --- a/assets/js/15.7a445b11.js +++ b/assets/js/15.74a14259.js @@ -1 +1 @@ -(window.webpackJsonp=window.webpackJsonp||[]).push([[15],{387:function(e,t,a){"use strict";a.r(t);var s=a(26),r=Object(s.a)({},(function(){var e=this,t=e.$createElement,a=e._self._c||t;return a("ContentSlotsDistributor",{attrs:{"slot-key":e.$parent.slotKey}},[a("h1",{attrs:{id:"page-frontmatter-title"}},[a("a",{staticClass:"header-anchor",attrs:{href:"#page-frontmatter-title"}},[e._v("#")]),e._v(" "+e._s(e.$page.frontmatter.title))]),e._v(" "),a("p",[e._v(e._s(e.$page.frontmatter.abstract))]),e._v(" "),a("MetadataTable"),e._v(" "),a("h2",{attrs:{id:"language"}},[a("a",{staticClass:"header-anchor",attrs:{href:"#language"}},[e._v("#")]),e._v(" Language")]),e._v(" "),a("Language"),e._v(" "),a("h2",{attrs:{id:"introduction"}},[a("a",{staticClass:"header-anchor",attrs:{href:"#introduction"}},[e._v("#")]),e._v(" Introduction")]),e._v(" "),a("p",[e._v("CSV Dialect defines a simple format to describe the various dialects of CSV files in a language agnostic manner. It aims to deal with a reasonably large subset of the features which differ between dialects, such as terminator strings, quoting rules, escape rules and so on. The specification has been modeled around the union of the csv modules in Python and Ruby, and the bulk load capabilities of MySQL and PostgresQL.")]),e._v(" "),a("h3",{attrs:{id:"excluded"}},[a("a",{staticClass:"header-anchor",attrs:{href:"#excluded"}},[e._v("#")]),e._v(" Excluded")]),e._v(" "),a("p",[e._v("CSV Dialect has nothing to do with the names, contents or types of the headers or data within the CSV file, only how it is formatted. However, CSV Dialect does allow the presence or absence of a header to be specified, similarly to "),a("a",{attrs:{href:"http://www.ietf.org/rfc/rfc4180.txt",target:"_blank",rel:"noopener noreferrer"}},[e._v("RFC4180"),a("OutboundLink")],1),e._v(".")]),e._v(" "),a("p",[e._v("CSV Dialect is also orthogonal to the character encoding used in the CSV file. Note that it is possible for files in CSV format to contain data in more than one encoding.")]),e._v(" "),a("h3",{attrs:{id:"usage"}},[a("a",{staticClass:"header-anchor",attrs:{href:"#usage"}},[e._v("#")]),e._v(" Usage")]),e._v(" "),a("p",[e._v("CSV Dialect is useful for programmes which might have to deal with multiple dialects of CSV file, but which can rely on being told out-of-band which dialect will be used in a given input stream. This reduces the need for heuristic inference of CSV dialects, and simplifies the implementation of CSV readers, which must juggle dialect inference, schema inference, unseekable input streams, character encoding issues, and the lazy reading of very large input streams.")]),e._v(" "),a("p",[e._v("Some related work can be found in "),a("a",{attrs:{href:"https://docs.google.com/spreadsheet/ccc?key=0AmU3V2vcPKrIdEhoU1NQSWtoQmJwcUNCelJtdkx2bFE&usp=sharing",target:"_blank",rel:"noopener noreferrer"}},[e._v("this comparison of csv dialect support"),a("OutboundLink")],1),e._v(", this "),a("a",{attrs:{href:"http://panda.readthedocs.org/en/latest/api.html#data-uploads",target:"_blank",rel:"noopener noreferrer"}},[e._v("example of similar JSON format"),a("OutboundLink")],1),e._v(", and in Python’s "),a("a",{attrs:{href:"http://www.python.org/dev/peps/pep-0305/",target:"_blank",rel:"noopener noreferrer"}},[e._v("PEP 305"),a("OutboundLink")],1),e._v(".")]),e._v(" "),a("h2",{attrs:{id:"specification"}},[a("a",{staticClass:"header-anchor",attrs:{href:"#specification"}},[e._v("#")]),e._v(" Specification")]),e._v(" "),a("p",[e._v("A CSV Dialect descriptor, "),a("code",[e._v("dialect")]),e._v(", "),a("code",[e._v("MUST")]),e._v(" be a JSON "),a("code",[e._v("object")]),e._v(" with the following properties:")]),e._v(" "),a("ul",[a("li",[a("code",[e._v("delimiter")]),e._v(" - specifies the character sequence which should separate fields (aka columns). Default = "),a("code",[e._v(",")]),e._v(". Example "),a("code",[e._v("\\t")]),e._v(". If not present, consumers should assume that it’s "),a("code",[e._v(",")]),e._v(".")]),e._v(" "),a("li",[a("code",[e._v("lineTerminator")]),e._v(" - specifies the character sequence which should terminate rows. Default = "),a("code",[e._v("\\r\\n")])]),e._v(" "),a("li",[a("code",[e._v("quoteChar")]),e._v(" - specifies a one-character string to use as the quoting character. Default = "),a("code",[e._v('"')])]),e._v(" "),a("li",[a("code",[e._v("doubleQuote")]),e._v(" - controls the handling of quotes inside fields. If true, two consecutive quotes should be interpreted as one. Default = "),a("code",[e._v("true")])]),e._v(" "),a("li",[a("code",[e._v("escapeChar")]),e._v(" - specifies a one-character string to use for escaping (for example, "),a("code",[e._v("\\")]),e._v("), mutually exclusive with "),a("code",[e._v("quoteChar")]),e._v(". Not set by default")]),e._v(" "),a("li",[a("code",[e._v("nullSequence")]),e._v(" - specifies the null sequence (for example "),a("code",[e._v("\\N")]),e._v("). Not set by default")]),e._v(" "),a("li",[a("code",[e._v("skipInitialSpace")]),e._v(" - specifies how to interpret whitespace which immediately follows a delimiter; if "),a("code",[e._v("false")]),e._v(", it means that whitespace immediately after a delimiter should be treated as part of the following field. Default = "),a("code",[e._v("false")])]),e._v(" "),a("li",[a("code",[e._v("header")]),e._v(" - indicates whether the file includes a header row. If "),a("code",[e._v("true")]),e._v(" the first row in the file is a header row, not data. Default = "),a("code",[e._v("true")])]),e._v(" "),a("li",[a("code",[e._v("commentChar")]),e._v(" - indicates a one-character string to ignore any line whose row begins with this character. Not set by default")]),e._v(" "),a("li",[a("code",[e._v("caseSensitiveHeader")]),e._v(" - indicates that case in the header is meaningful. For example, columns "),a("code",[e._v("CAT")]),e._v(" and "),a("code",[e._v("Cat")]),e._v(" should not be equated. Default = "),a("code",[e._v("false")])]),e._v(" "),a("li",[a("code",[e._v("csvddfVersion")]),e._v(" - a number, in n.n format, e.g., "),a("code",[e._v("1.2")]),e._v(". If not present, consumers should assume latest schema version.")])]),e._v(" "),a("h3",{attrs:{id:"example"}},[a("a",{staticClass:"header-anchor",attrs:{href:"#example"}},[e._v("#")]),e._v(" Example")]),e._v(" "),a("p",[e._v("Here’s an example:")]),e._v(" "),a("div",{staticClass:"language-json extra-class"},[a("pre",{pre:!0,attrs:{class:"language-json"}},[a("code",[a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("{")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token property"}},[e._v('"dialect"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("{")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token property"}},[e._v('"csvddfVersion"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token number"}},[e._v("1.2")]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v(",")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token property"}},[e._v('"delimiter"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('";"')]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v(",")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token property"}},[e._v('"doubleQuote"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token boolean"}},[e._v("true")]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v(",")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token property"}},[e._v('"lineTerminator"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"\\r\\n"')]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v(",")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token property"}},[e._v('"quoteChar"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"\\""')]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v(",")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token property"}},[e._v('"skipInitialSpace"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token boolean"}},[e._v("true")]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v(",")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token property"}},[e._v('"header"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token boolean"}},[e._v("true")]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v(",")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token property"}},[e._v('"commentChar"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"#"')]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("}")]),e._v("\n"),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("}")]),e._v("\n")])])])],1)}),[],!1,null,null,null);t.default=r.exports}}]); \ No newline at end of file +(window.webpackJsonp=window.webpackJsonp||[]).push([[15],{385:function(e,t,a){"use strict";a.r(t);var s=a(26),r=Object(s.a)({},(function(){var e=this,t=e.$createElement,a=e._self._c||t;return a("ContentSlotsDistributor",{attrs:{"slot-key":e.$parent.slotKey}},[a("h1",{attrs:{id:"page-frontmatter-title"}},[a("a",{staticClass:"header-anchor",attrs:{href:"#page-frontmatter-title"}},[e._v("#")]),e._v(" "+e._s(e.$page.frontmatter.title))]),e._v(" "),a("p",[e._v(e._s(e.$page.frontmatter.abstract))]),e._v(" "),a("MetadataTable"),e._v(" "),a("h2",{attrs:{id:"language"}},[a("a",{staticClass:"header-anchor",attrs:{href:"#language"}},[e._v("#")]),e._v(" Language")]),e._v(" "),a("Language"),e._v(" "),a("h2",{attrs:{id:"introduction"}},[a("a",{staticClass:"header-anchor",attrs:{href:"#introduction"}},[e._v("#")]),e._v(" Introduction")]),e._v(" "),a("p",[e._v("CSV Dialect defines a simple format to describe the various dialects of CSV files in a language agnostic manner. It aims to deal with a reasonably large subset of the features which differ between dialects, such as terminator strings, quoting rules, escape rules and so on. The specification has been modeled around the union of the csv modules in Python and Ruby, and the bulk load capabilities of MySQL and PostgresQL.")]),e._v(" "),a("h3",{attrs:{id:"excluded"}},[a("a",{staticClass:"header-anchor",attrs:{href:"#excluded"}},[e._v("#")]),e._v(" Excluded")]),e._v(" "),a("p",[e._v("CSV Dialect has nothing to do with the names, contents or types of the headers or data within the CSV file, only how it is formatted. However, CSV Dialect does allow the presence or absence of a header to be specified, similarly to "),a("a",{attrs:{href:"http://www.ietf.org/rfc/rfc4180.txt",target:"_blank",rel:"noopener noreferrer"}},[e._v("RFC4180"),a("OutboundLink")],1),e._v(".")]),e._v(" "),a("p",[e._v("CSV Dialect is also orthogonal to the character encoding used in the CSV file. Note that it is possible for files in CSV format to contain data in more than one encoding.")]),e._v(" "),a("h3",{attrs:{id:"usage"}},[a("a",{staticClass:"header-anchor",attrs:{href:"#usage"}},[e._v("#")]),e._v(" Usage")]),e._v(" "),a("p",[e._v("CSV Dialect is useful for programmes which might have to deal with multiple dialects of CSV file, but which can rely on being told out-of-band which dialect will be used in a given input stream. This reduces the need for heuristic inference of CSV dialects, and simplifies the implementation of CSV readers, which must juggle dialect inference, schema inference, unseekable input streams, character encoding issues, and the lazy reading of very large input streams.")]),e._v(" "),a("p",[e._v("Some related work can be found in "),a("a",{attrs:{href:"https://docs.google.com/spreadsheet/ccc?key=0AmU3V2vcPKrIdEhoU1NQSWtoQmJwcUNCelJtdkx2bFE&usp=sharing",target:"_blank",rel:"noopener noreferrer"}},[e._v("this comparison of csv dialect support"),a("OutboundLink")],1),e._v(", this "),a("a",{attrs:{href:"http://panda.readthedocs.org/en/latest/api.html#data-uploads",target:"_blank",rel:"noopener noreferrer"}},[e._v("example of similar JSON format"),a("OutboundLink")],1),e._v(", and in Python’s "),a("a",{attrs:{href:"http://www.python.org/dev/peps/pep-0305/",target:"_blank",rel:"noopener noreferrer"}},[e._v("PEP 305"),a("OutboundLink")],1),e._v(".")]),e._v(" "),a("h2",{attrs:{id:"specification"}},[a("a",{staticClass:"header-anchor",attrs:{href:"#specification"}},[e._v("#")]),e._v(" Specification")]),e._v(" "),a("p",[e._v("A CSV Dialect descriptor, "),a("code",[e._v("dialect")]),e._v(", "),a("code",[e._v("MUST")]),e._v(" be a JSON "),a("code",[e._v("object")]),e._v(" with the following properties:")]),e._v(" "),a("ul",[a("li",[a("code",[e._v("delimiter")]),e._v(" - specifies the character sequence which should separate fields (aka columns). Default = "),a("code",[e._v(",")]),e._v(". Example "),a("code",[e._v("\\t")]),e._v(". If not present, consumers should assume that it’s "),a("code",[e._v(",")]),e._v(".")]),e._v(" "),a("li",[a("code",[e._v("lineTerminator")]),e._v(" - specifies the character sequence which should terminate rows. Default = "),a("code",[e._v("\\r\\n")])]),e._v(" "),a("li",[a("code",[e._v("quoteChar")]),e._v(" - specifies a one-character string to use as the quoting character. Default = "),a("code",[e._v('"')])]),e._v(" "),a("li",[a("code",[e._v("doubleQuote")]),e._v(" - controls the handling of quotes inside fields. If true, two consecutive quotes should be interpreted as one. Default = "),a("code",[e._v("true")])]),e._v(" "),a("li",[a("code",[e._v("escapeChar")]),e._v(" - specifies a one-character string to use for escaping (for example, "),a("code",[e._v("\\")]),e._v("), mutually exclusive with "),a("code",[e._v("quoteChar")]),e._v(". Not set by default")]),e._v(" "),a("li",[a("code",[e._v("nullSequence")]),e._v(" - specifies the null sequence (for example "),a("code",[e._v("\\N")]),e._v("). Not set by default")]),e._v(" "),a("li",[a("code",[e._v("skipInitialSpace")]),e._v(" - specifies how to interpret whitespace which immediately follows a delimiter; if "),a("code",[e._v("false")]),e._v(", it means that whitespace immediately after a delimiter should be treated as part of the following field. Default = "),a("code",[e._v("false")])]),e._v(" "),a("li",[a("code",[e._v("header")]),e._v(" - indicates whether the file includes a header row. If "),a("code",[e._v("true")]),e._v(" the first row in the file is a header row, not data. Default = "),a("code",[e._v("true")])]),e._v(" "),a("li",[a("code",[e._v("commentChar")]),e._v(" - indicates a one-character string to ignore any line whose row begins with this character. Not set by default")]),e._v(" "),a("li",[a("code",[e._v("caseSensitiveHeader")]),e._v(" - indicates that case in the header is meaningful. For example, columns "),a("code",[e._v("CAT")]),e._v(" and "),a("code",[e._v("Cat")]),e._v(" should not be equated. Default = "),a("code",[e._v("false")])]),e._v(" "),a("li",[a("code",[e._v("csvddfVersion")]),e._v(" - a number, in n.n format, e.g., "),a("code",[e._v("1.2")]),e._v(". If not present, consumers should assume latest schema version.")])]),e._v(" "),a("h3",{attrs:{id:"example"}},[a("a",{staticClass:"header-anchor",attrs:{href:"#example"}},[e._v("#")]),e._v(" Example")]),e._v(" "),a("p",[e._v("Here’s an example:")]),e._v(" "),a("div",{staticClass:"language-json extra-class"},[a("pre",{pre:!0,attrs:{class:"language-json"}},[a("code",[a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("{")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token property"}},[e._v('"dialect"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("{")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token property"}},[e._v('"csvddfVersion"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token number"}},[e._v("1.2")]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v(",")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token property"}},[e._v('"delimiter"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('";"')]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v(",")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token property"}},[e._v('"doubleQuote"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token boolean"}},[e._v("true")]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v(",")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token property"}},[e._v('"lineTerminator"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"\\r\\n"')]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v(",")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token property"}},[e._v('"quoteChar"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"\\""')]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v(",")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token property"}},[e._v('"skipInitialSpace"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token boolean"}},[e._v("true")]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v(",")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token property"}},[e._v('"header"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token boolean"}},[e._v("true")]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v(",")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token property"}},[e._v('"commentChar"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"#"')]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("}")]),e._v("\n"),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("}")]),e._v("\n")])])])],1)}),[],!1,null,null,null);t.default=r.exports}}]); \ No newline at end of file diff --git a/assets/js/17.4c336f4b.js b/assets/js/17.4427245e.js similarity index 99% rename from assets/js/17.4c336f4b.js rename to assets/js/17.4427245e.js index bea7daf1..2fa01e53 100644 --- a/assets/js/17.4c336f4b.js +++ b/assets/js/17.4427245e.js @@ -1 +1 @@ -(window.webpackJsonp=window.webpackJsonp||[]).push([[17],{385:function(t,a,e){"use strict";e.r(a);var s=e(26),r=Object(s.a)({},(function(){var t=this,a=t.$createElement,e=t._self._c||a;return e("ContentSlotsDistributor",{attrs:{"slot-key":t.$parent.slotKey}},[e("h1",{attrs:{id:"page-frontmatter-title"}},[e("a",{staticClass:"header-anchor",attrs:{href:"#page-frontmatter-title"}},[t._v("#")]),t._v(" "+t._s(t.$page.frontmatter.title))]),t._v(" "),e("p",[t._v(t._s(t.$page.frontmatter.abstract))]),t._v(" "),e("MetadataTable"),t._v(" "),e("h2",{attrs:{id:"language"}},[e("a",{staticClass:"header-anchor",attrs:{href:"#language"}},[t._v("#")]),t._v(" Language")]),t._v(" "),e("Language"),t._v(" "),e("h2",{attrs:{id:"introduction"}},[e("a",{staticClass:"header-anchor",attrs:{href:"#introduction"}},[t._v("#")]),t._v(" Introduction")]),t._v(" "),e("p",[t._v("A Data Package consists of:")]),t._v(" "),e("ul",[e("li",[t._v("Metadata that describes the structure and contents of the package")]),t._v(" "),e("li",[t._v("Resources such as data files that form the contents of the package")])]),t._v(" "),e("p",[t._v("The Data Package metadata is stored in a “descriptor”. This descriptor is what makes a collection of data a Data Package. The structure of this descriptor is the main content of the specification below.")]),t._v(" "),e("p",[t._v("In addition to this descriptor a data package will include other resources such as data files. The Data Package specification does NOT impose any requirements on their form or structure and can therefore be used for packaging "),e("strong",[t._v("any kind of data")]),t._v(".")]),t._v(" "),e("p",[t._v("The data included in the package may be provided as:")]),t._v(" "),e("ul",[e("li",[t._v("Files bundled locally with the package descriptor")]),t._v(" "),e("li",[t._v("Remote resources, referenced by URL")]),t._v(" "),e("li",[t._v("“Inline” data (see below) which is included directly in the descriptor")])]),t._v(" "),e("h3",{attrs:{id:"illustrative-structure"}},[e("a",{staticClass:"header-anchor",attrs:{href:"#illustrative-structure"}},[t._v("#")]),t._v(" Illustrative Structure")]),t._v(" "),e("p",[t._v("A minimal data package on disk would be a directory containing a single file:")]),t._v(" "),e("div",{staticClass:"language- extra-class"},[e("pre",{pre:!0,attrs:{class:"language-text"}},[e("code",[t._v("datapackage.json # (required) metadata and schemas for this data package\n")])])]),e("p",[t._v("Lacking a single external source of data would make this of limited use. A slightly less minimal version would be:")]),t._v(" "),e("div",{staticClass:"language- extra-class"},[e("pre",{pre:!0,attrs:{class:"language-text"}},[e("code",[t._v("datapackage.json\n# a data file (CSV in this case)\ndata.csv\n")])])]),e("p",[t._v("Additional files such as a README, scripts (for processing or analyzing the data) and other material may be provided. By convention scripts go in a scripts directory and thus, a more elaborate data package could look like this:")]),t._v(" "),e("div",{staticClass:"language- extra-class"},[e("pre",{pre:!0,attrs:{class:"language-text"}},[e("code",[t._v("datapackage.json # (required) metadata and schemas for this data package\nREADME.md # (optional) README in markdown format\n\n# data files may go either in data subdirectory or in main directory\nmydata.csv\ndata/otherdata.csv\n\n# the directory for code scripts - again these can go in the base directory\nscripts/my-preparation-script.py\n")])])]),e("p",[t._v("Several example data packages can be found in the "),e("a",{attrs:{href:"https://github.com/datasets",target:"_blank",rel:"noopener noreferrer"}},[t._v("datasets organization on github"),e("OutboundLink")],1),t._v(", including:")]),t._v(" "),e("ul",[e("li",[e("a",{attrs:{href:"https://github.com/datasets/gdp",target:"_blank",rel:"noopener noreferrer"}},[t._v("World GDP"),e("OutboundLink")],1)]),t._v(" "),e("li",[e("a",{attrs:{href:"https://github.com/datasets/country-codes",target:"_blank",rel:"noopener noreferrer"}},[t._v("ISO 3166-2 country codes"),e("OutboundLink")],1)])]),t._v(" "),e("h2",{attrs:{id:"specification"}},[e("a",{staticClass:"header-anchor",attrs:{href:"#specification"}},[t._v("#")]),t._v(" Specification")]),t._v(" "),e("h3",{attrs:{id:"descriptor"}},[e("a",{staticClass:"header-anchor",attrs:{href:"#descriptor"}},[t._v("#")]),t._v(" Descriptor")]),t._v(" "),e("p",[t._v("The descriptor is the central file in a Data Package. It provides:")]),t._v(" "),e("ul",[e("li",[t._v("General metadata such as the package’s title, license, publisher etc")]),t._v(" "),e("li",[t._v("A list of the data “resources” that make up the package including their location on disk or online and other relevant information (including, possibly, schema information about these data resources in a structured form)")])]),t._v(" "),e("p",[t._v("A Data Package descriptor "),e("code",[t._v("MUST")]),t._v(" be a valid JSON "),e("code",[t._v("object")]),t._v(". (JSON is defined in "),e("a",{attrs:{href:"http://www.ietf.org/rfc/rfc4627.txt",target:"_blank",rel:"noopener noreferrer"}},[t._v("RFC 4627"),e("OutboundLink")],1),t._v("). When available as a file it "),e("code",[t._v("MUST")]),t._v(" be named "),e("code",[t._v("datapackage.json")]),t._v(" and it "),e("code",[t._v("MUST")]),t._v(" be placed in the top-level directory (relative to any other resources provided as part of the data package).")]),t._v(" "),e("p",[t._v("The descriptor "),e("code",[t._v("MUST")]),t._v(" contain a "),e("code",[t._v("resources")]),t._v(" property describing the data resources.")]),t._v(" "),e("p",[t._v("All other properties are considered "),e("code",[t._v("metadata")]),t._v(" properties. The descriptor "),e("code",[t._v("MAY")]),t._v(" contain any number of other "),e("code",[t._v("metadata")]),t._v(" properties. The following sections provides a description of required and optional metadata properties for a Data Package descriptor.")]),t._v(" "),e("p",[t._v("Adherence to the specification does not imply that additional, non-specified properties cannot be used: a descriptor "),e("code",[t._v("MAY")]),t._v(" include any number of properties in additional to those described as required and optional properties. For example, if you were storing time series data and wanted to list the temporal coverage of the data in the Data Package you could add a property "),e("code",[t._v("temporal")]),t._v(" (cf "),e("a",{attrs:{href:"http://dublincore.org/documents/usageguide/qualifiers.shtml#temporal",target:"_blank",rel:"noopener noreferrer"}},[t._v("Dublin Core"),e("OutboundLink")],1),t._v("):")]),t._v(" "),e("div",{staticClass:"language-javascript extra-class"},[e("pre",{pre:!0,attrs:{class:"language-javascript"}},[e("code",[e("span",{pre:!0,attrs:{class:"token string"}},[t._v('"temporal"')]),e("span",{pre:!0,attrs:{class:"token operator"}},[t._v(":")]),t._v(" "),e("span",{pre:!0,attrs:{class:"token punctuation"}},[t._v("{")]),t._v("\n "),e("span",{pre:!0,attrs:{class:"token string"}},[t._v('"name"')]),e("span",{pre:!0,attrs:{class:"token operator"}},[t._v(":")]),t._v(" "),e("span",{pre:!0,attrs:{class:"token string"}},[t._v('"19th Century"')]),e("span",{pre:!0,attrs:{class:"token punctuation"}},[t._v(",")]),t._v("\n "),e("span",{pre:!0,attrs:{class:"token string"}},[t._v('"start"')]),e("span",{pre:!0,attrs:{class:"token operator"}},[t._v(":")]),t._v(" "),e("span",{pre:!0,attrs:{class:"token string"}},[t._v('"1800-01-01"')]),e("span",{pre:!0,attrs:{class:"token punctuation"}},[t._v(",")]),t._v("\n "),e("span",{pre:!0,attrs:{class:"token string"}},[t._v('"end"')]),e("span",{pre:!0,attrs:{class:"token operator"}},[t._v(":")]),t._v(" "),e("span",{pre:!0,attrs:{class:"token string"}},[t._v('"1899-12-31"')]),t._v("\n"),e("span",{pre:!0,attrs:{class:"token punctuation"}},[t._v("}")]),t._v("\n")])])]),e("p",[t._v("This flexibility enables specific communities to extend Data Packages as appropriate for the data they manage. As an example, the "),e("RouterLink",{attrs:{to:"/tabular-data-package/"}},[t._v("Tabular Data Package")]),t._v(" specification extends Data Package to the case where all the data is tabular and stored in CSV.")],1),t._v(" "),e("p",[t._v("Here is an illustrative example of a datapackage JSON file:")]),t._v(" "),e("div",{staticClass:"language-javascript extra-class"},[e("pre",{pre:!0,attrs:{class:"language-javascript"}},[e("code",[e("span",{pre:!0,attrs:{class:"token punctuation"}},[t._v("{")]),t._v("\n # general "),e("span",{pre:!0,attrs:{class:"token string"}},[t._v('"metadata"')]),t._v(" like title"),e("span",{pre:!0,attrs:{class:"token punctuation"}},[t._v(",")]),t._v(" sources etc\n "),e("span",{pre:!0,attrs:{class:"token string"}},[t._v('"name"')]),t._v(" "),e("span",{pre:!0,attrs:{class:"token operator"}},[t._v(":")]),t._v(" "),e("span",{pre:!0,attrs:{class:"token string"}},[t._v('"a-unique-human-readable-and-url-usable-identifier"')]),e("span",{pre:!0,attrs:{class:"token punctuation"}},[t._v(",")]),t._v("\n "),e("span",{pre:!0,attrs:{class:"token string"}},[t._v('"title"')]),t._v(" "),e("span",{pre:!0,attrs:{class:"token operator"}},[t._v(":")]),t._v(" "),e("span",{pre:!0,attrs:{class:"token string"}},[t._v('"A nice title"')]),e("span",{pre:!0,attrs:{class:"token punctuation"}},[t._v(",")]),t._v("\n "),e("span",{pre:!0,attrs:{class:"token string"}},[t._v('"licenses"')]),t._v(" "),e("span",{pre:!0,attrs:{class:"token operator"}},[t._v(":")]),t._v(" "),e("span",{pre:!0,attrs:{class:"token punctuation"}},[t._v("[")]),t._v(" "),e("span",{pre:!0,attrs:{class:"token operator"}},[t._v("...")]),t._v(" "),e("span",{pre:!0,attrs:{class:"token punctuation"}},[t._v("]")]),e("span",{pre:!0,attrs:{class:"token punctuation"}},[t._v(",")]),t._v("\n "),e("span",{pre:!0,attrs:{class:"token string"}},[t._v('"sources"')]),t._v(" "),e("span",{pre:!0,attrs:{class:"token operator"}},[t._v(":")]),t._v(" "),e("span",{pre:!0,attrs:{class:"token punctuation"}},[t._v("[")]),e("span",{pre:!0,attrs:{class:"token operator"}},[t._v("...")]),e("span",{pre:!0,attrs:{class:"token punctuation"}},[t._v("]")]),e("span",{pre:!0,attrs:{class:"token punctuation"}},[t._v(",")]),t._v("\n # list "),e("span",{pre:!0,attrs:{class:"token keyword"}},[t._v("of")]),t._v(" the data resources "),e("span",{pre:!0,attrs:{class:"token keyword"}},[t._v("in")]),t._v(" "),e("span",{pre:!0,attrs:{class:"token keyword"}},[t._v("this")]),t._v(" data "),e("span",{pre:!0,attrs:{class:"token keyword"}},[t._v("package")]),t._v("\n "),e("span",{pre:!0,attrs:{class:"token string"}},[t._v('"resources"')]),e("span",{pre:!0,attrs:{class:"token operator"}},[t._v(":")]),t._v(" "),e("span",{pre:!0,attrs:{class:"token punctuation"}},[t._v("[")]),t._v("\n "),e("span",{pre:!0,attrs:{class:"token punctuation"}},[t._v("{")]),t._v("\n "),e("span",{pre:!0,attrs:{class:"token operator"}},[t._v("...")]),t._v(" resource info described below "),e("span",{pre:!0,attrs:{class:"token operator"}},[t._v("...")]),t._v("\n "),e("span",{pre:!0,attrs:{class:"token punctuation"}},[t._v("}")]),t._v("\n "),e("span",{pre:!0,attrs:{class:"token punctuation"}},[t._v("]")]),e("span",{pre:!0,attrs:{class:"token punctuation"}},[t._v(",")]),t._v("\n # optional\n "),e("span",{pre:!0,attrs:{class:"token operator"}},[t._v("...")]),t._v(" additional information "),e("span",{pre:!0,attrs:{class:"token operator"}},[t._v("...")]),t._v("\n"),e("span",{pre:!0,attrs:{class:"token punctuation"}},[t._v("}")]),t._v("\n")])])]),e("h3",{attrs:{id:"resource-information"}},[e("a",{staticClass:"header-anchor",attrs:{href:"#resource-information"}},[t._v("#")]),t._v(" Resource Information")]),t._v(" "),e("p",[t._v("Packaged data resources are described in the "),e("code",[t._v("resources")]),t._v(" property of the package descriptor. This property "),e("code",[t._v("MUST")]),t._v(" be an array of "),e("code",[t._v("objects")]),t._v(". Each object "),e("code",[t._v("MUST")]),t._v(" follow the "),e("RouterLink",{attrs:{to:"/data-resource/"}},[t._v("Data Resource specification")]),t._v(".")],1),t._v(" "),e("h3",{attrs:{id:"metadata"}},[e("a",{staticClass:"header-anchor",attrs:{href:"#metadata"}},[t._v("#")]),t._v(" Metadata")]),t._v(" "),e("h4",{attrs:{id:"required-properties"}},[e("a",{staticClass:"header-anchor",attrs:{href:"#required-properties"}},[t._v("#")]),t._v(" Required Properties")]),t._v(" "),e("p",[t._v("The "),e("code",[t._v("resources")]),t._v(" property is required, with at least one resource.")]),t._v(" "),e("h4",{attrs:{id:"recommended-properties"}},[e("a",{staticClass:"header-anchor",attrs:{href:"#recommended-properties"}},[t._v("#")]),t._v(" Recommended Properties")]),t._v(" "),e("p",[t._v("In addition to the required properties, the following properties "),e("code",[t._v("SHOULD")]),t._v(" be included in every package descriptor:")]),t._v(" "),e("h5",{attrs:{id:"name"}},[e("a",{staticClass:"header-anchor",attrs:{href:"#name"}},[t._v("#")]),t._v(" "),e("code",[t._v("name")])]),t._v(" "),e("p",[t._v("A short url-usable (and preferably human-readable) name of the package. This "),e("code",[t._v("MUST")]),t._v(" be lower-case and contain only alphanumeric characters along with “.”, “_” or “-” characters. It will function as a unique identifier and therefore "),e("code",[t._v("SHOULD")]),t._v(" be unique in relation to any registry in which this package will be deposited (and preferably globally unique).")]),t._v(" "),e("p",[t._v("The name "),e("code",[t._v("SHOULD")]),t._v(" be invariant, meaning that it "),e("code",[t._v("SHOULD NOT")]),t._v(" change when a data package is updated, unless the new package version should be considered a distinct package, e.g. due to significant changes in structure or interpretation. Version distinction "),e("code",[t._v("SHOULD")]),t._v(" be left to the version property. As a corollary, the name also "),e("code",[t._v("SHOULD NOT")]),t._v(" include an indication of time range covered.")]),t._v(" "),e("h5",{attrs:{id:"id"}},[e("a",{staticClass:"header-anchor",attrs:{href:"#id"}},[t._v("#")]),t._v(" "),e("code",[t._v("id")])]),t._v(" "),e("p",[t._v("A property reserved for globally unique identifiers. Examples of identifiers that are unique include UUIDs and DOIs.")]),t._v(" "),e("p",[t._v("A common usage pattern for Data Packages is as a packaging format within the bounds of a system or platform. In these cases, a unique identifier for a package is desired for common data handling workflows, such as updating an existing package. While at the level of the specification, global uniqueness cannot be validated, consumers using the "),e("code",[t._v("id")]),t._v(" property "),e("code",[t._v("MUST")]),t._v(" ensure identifiers are globally unique.")]),t._v(" "),e("p",[t._v("Examples:")]),t._v(" "),e("div",{staticClass:"language-javascript extra-class"},[e("pre",{pre:!0,attrs:{class:"language-javascript"}},[e("code",[e("span",{pre:!0,attrs:{class:"token punctuation"}},[t._v("{")]),t._v("\n "),e("span",{pre:!0,attrs:{class:"token string"}},[t._v('"id"')]),e("span",{pre:!0,attrs:{class:"token operator"}},[t._v(":")]),t._v(" "),e("span",{pre:!0,attrs:{class:"token string"}},[t._v('"b03ec84-77fd-4270-813b-0c698943f7ce"')]),t._v("\n"),e("span",{pre:!0,attrs:{class:"token punctuation"}},[t._v("}")]),t._v("\n")])])]),e("div",{staticClass:"language-javascript extra-class"},[e("pre",{pre:!0,attrs:{class:"language-javascript"}},[e("code",[e("span",{pre:!0,attrs:{class:"token punctuation"}},[t._v("{")]),t._v("\n "),e("span",{pre:!0,attrs:{class:"token string"}},[t._v('"id"')]),e("span",{pre:!0,attrs:{class:"token operator"}},[t._v(":")]),t._v(" "),e("span",{pre:!0,attrs:{class:"token string"}},[t._v('"https://doi.org/10.1594/PANGAEA.726855"')]),t._v("\n"),e("span",{pre:!0,attrs:{class:"token punctuation"}},[t._v("}")]),t._v("\n")])])]),e("h5",{attrs:{id:"licenses"}},[e("a",{staticClass:"header-anchor",attrs:{href:"#licenses"}},[t._v("#")]),t._v(" "),e("code",[t._v("licenses")])]),t._v(" "),e("p",[t._v("The license(s) under which the package is provided.")]),t._v(" "),e("p",[e("strong",[t._v("This property is not legally binding and does not guarantee the package is licensed under the terms defined in this property.")])]),t._v(" "),e("p",[e("code",[t._v("licenses")]),t._v(" "),e("code",[t._v("MUST")]),t._v(" be an array. Each item in the array is a License. Each "),e("code",[t._v("MUST")]),t._v(" be an "),e("code",[t._v("object")]),t._v(". The object "),e("code",[t._v("MUST")]),t._v(" contain a "),e("code",[t._v("name")]),t._v(" property and/or a "),e("code",[t._v("path")]),t._v(" property. It "),e("code",[t._v("MAY")]),t._v(" contain a "),e("code",[t._v("title")]),t._v(" property.")]),t._v(" "),e("p",[t._v("Here is an example:")]),t._v(" "),e("div",{staticClass:"language-javascript extra-class"},[e("pre",{pre:!0,attrs:{class:"language-javascript"}},[e("code",[e("span",{pre:!0,attrs:{class:"token string"}},[t._v('"licenses"')]),e("span",{pre:!0,attrs:{class:"token operator"}},[t._v(":")]),t._v(" "),e("span",{pre:!0,attrs:{class:"token punctuation"}},[t._v("[")]),e("span",{pre:!0,attrs:{class:"token punctuation"}},[t._v("{")]),t._v("\n "),e("span",{pre:!0,attrs:{class:"token string"}},[t._v('"name"')]),e("span",{pre:!0,attrs:{class:"token operator"}},[t._v(":")]),t._v(" "),e("span",{pre:!0,attrs:{class:"token string"}},[t._v('"ODC-PDDL-1.0"')]),e("span",{pre:!0,attrs:{class:"token punctuation"}},[t._v(",")]),t._v("\n "),e("span",{pre:!0,attrs:{class:"token string"}},[t._v('"path"')]),e("span",{pre:!0,attrs:{class:"token operator"}},[t._v(":")]),t._v(" "),e("span",{pre:!0,attrs:{class:"token string"}},[t._v('"http://opendatacommons.org/licenses/pddl/"')]),e("span",{pre:!0,attrs:{class:"token punctuation"}},[t._v(",")]),t._v("\n "),e("span",{pre:!0,attrs:{class:"token string"}},[t._v('"title"')]),e("span",{pre:!0,attrs:{class:"token operator"}},[t._v(":")]),t._v(" "),e("span",{pre:!0,attrs:{class:"token string"}},[t._v('"Open Data Commons Public Domain Dedication and License v1.0"')]),t._v("\n"),e("span",{pre:!0,attrs:{class:"token punctuation"}},[t._v("}")]),e("span",{pre:!0,attrs:{class:"token punctuation"}},[t._v("]")]),t._v("\n")])])]),e("ul",[e("li",[e("code",[t._v("name")]),t._v(": The "),e("code",[t._v("name")]),t._v(" "),e("code",[t._v("MUST")]),t._v(" be an "),e("a",{attrs:{href:"http://licenses.opendefinition.org/",target:"_blank",rel:"noopener noreferrer"}},[t._v("Open Definition license ID"),e("OutboundLink")],1)]),t._v(" "),e("li",[e("code",[t._v("path")]),t._v(": A "),e("RouterLink",{attrs:{to:"/data-resource/#url-or-path"}},[t._v("url-or-path")]),t._v(" string, that is a fully qualified HTTP address, or a relative POSIX path (see "),e("RouterLink",{attrs:{to:"/data-resource/#url-or-path"}},[t._v("the url-or-path definition in Data Resource for details")]),t._v(").")],1),t._v(" "),e("li",[e("code",[t._v("title")]),t._v(": A human-readable title.")])]),t._v(" "),e("h5",{attrs:{id:"profile"}},[e("a",{staticClass:"header-anchor",attrs:{href:"#profile"}},[t._v("#")]),t._v(" "),e("code",[t._v("profile")])]),t._v(" "),e("p",[t._v("A string identifying the "),e("RouterLink",{attrs:{to:"/profiles/"}},[t._v("profile")]),t._v(" of this descriptor as per the "),e("RouterLink",{attrs:{to:"/profiles/"}},[t._v("profiles")]),t._v(" specification.")],1),t._v(" "),e("p",[t._v("Examples:")]),t._v(" "),e("div",{staticClass:"language-javascript extra-class"},[e("pre",{pre:!0,attrs:{class:"language-javascript"}},[e("code",[e("span",{pre:!0,attrs:{class:"token punctuation"}},[t._v("{")]),t._v("\n "),e("span",{pre:!0,attrs:{class:"token string"}},[t._v('"profile"')]),e("span",{pre:!0,attrs:{class:"token operator"}},[t._v(":")]),t._v(" "),e("span",{pre:!0,attrs:{class:"token string"}},[t._v('"tabular-data-package"')]),t._v("\n"),e("span",{pre:!0,attrs:{class:"token punctuation"}},[t._v("}")]),t._v("\n")])])]),e("div",{staticClass:"language-javascript extra-class"},[e("pre",{pre:!0,attrs:{class:"language-javascript"}},[e("code",[e("span",{pre:!0,attrs:{class:"token punctuation"}},[t._v("{")]),t._v("\n "),e("span",{pre:!0,attrs:{class:"token string"}},[t._v('"profile"')]),e("span",{pre:!0,attrs:{class:"token operator"}},[t._v(":")]),t._v(" "),e("span",{pre:!0,attrs:{class:"token string"}},[t._v('"http://example.com/my-profiles-json-schema.json"')]),t._v("\n"),e("span",{pre:!0,attrs:{class:"token punctuation"}},[t._v("}")]),t._v("\n")])])]),e("h4",{attrs:{id:"optional-properties"}},[e("a",{staticClass:"header-anchor",attrs:{href:"#optional-properties"}},[t._v("#")]),t._v(" Optional Properties")]),t._v(" "),e("p",[t._v("The following are commonly used properties that the package descriptor "),e("code",[t._v("MAY")]),t._v(" contain:")]),t._v(" "),e("h5",{attrs:{id:"title"}},[e("a",{staticClass:"header-anchor",attrs:{href:"#title"}},[t._v("#")]),t._v(" "),e("code",[t._v("title")])]),t._v(" "),e("p",[t._v("A "),e("code",[t._v("string")]),t._v(" providing a title or one sentence description for this package")]),t._v(" "),e("h5",{attrs:{id:"description"}},[e("a",{staticClass:"header-anchor",attrs:{href:"#description"}},[t._v("#")]),t._v(" "),e("code",[t._v("description")])]),t._v(" "),e("p",[t._v("A description of the package. The description "),e("code",[t._v("MUST")]),t._v(" be "),e("a",{attrs:{href:"http://commonmark.org/",target:"_blank",rel:"noopener noreferrer"}},[t._v("markdown"),e("OutboundLink")],1),t._v(" formatted – this also allows for simple plain text as plain text is itself valid markdown. The first paragraph (up to the first double line break) should be usable as summary information for the package.")]),t._v(" "),e("h5",{attrs:{id:"homepage"}},[e("a",{staticClass:"header-anchor",attrs:{href:"#homepage"}},[t._v("#")]),t._v(" "),e("code",[t._v("homepage")])]),t._v(" "),e("p",[t._v("A URL for the home on the web that is related to this data package.")]),t._v(" "),e("h5",{attrs:{id:"version"}},[e("a",{staticClass:"header-anchor",attrs:{href:"#version"}},[t._v("#")]),t._v(" "),e("code",[t._v("version")])]),t._v(" "),e("p",[t._v("A version string identifying the version of the package. It should conform to the "),e("a",{attrs:{href:"http://semver.org",target:"_blank",rel:"noopener noreferrer"}},[t._v("Semantic Versioning"),e("OutboundLink")],1),t._v(" requirements and should follow the "),e("RouterLink",{attrs:{to:"/patterns/#data-package-version"}},[t._v("Data Package Version")]),t._v(" pattern.")],1),t._v(" "),e("h5",{attrs:{id:"sources"}},[e("a",{staticClass:"header-anchor",attrs:{href:"#sources"}},[t._v("#")]),t._v(" "),e("code",[t._v("sources")])]),t._v(" "),e("p",[t._v("The raw sources for this data package. It "),e("code",[t._v("MUST")]),t._v(" be an array of Source objects. Each Source object "),e("code",[t._v("MUST")]),t._v(" have a "),e("code",[t._v("title")]),t._v(" and "),e("code",[t._v("MAY")]),t._v(" have "),e("code",[t._v("path")]),t._v(" and/or "),e("code",[t._v("email")]),t._v(" properties. Example:")]),t._v(" "),e("div",{staticClass:"language-javascript extra-class"},[e("pre",{pre:!0,attrs:{class:"language-javascript"}},[e("code",[e("span",{pre:!0,attrs:{class:"token string"}},[t._v('"sources"')]),e("span",{pre:!0,attrs:{class:"token operator"}},[t._v(":")]),t._v(" "),e("span",{pre:!0,attrs:{class:"token punctuation"}},[t._v("[")]),e("span",{pre:!0,attrs:{class:"token punctuation"}},[t._v("{")]),t._v("\n "),e("span",{pre:!0,attrs:{class:"token string"}},[t._v('"title"')]),e("span",{pre:!0,attrs:{class:"token operator"}},[t._v(":")]),t._v(" "),e("span",{pre:!0,attrs:{class:"token string"}},[t._v('"World Bank and OECD"')]),e("span",{pre:!0,attrs:{class:"token punctuation"}},[t._v(",")]),t._v("\n "),e("span",{pre:!0,attrs:{class:"token string"}},[t._v('"path"')]),e("span",{pre:!0,attrs:{class:"token operator"}},[t._v(":")]),t._v(" "),e("span",{pre:!0,attrs:{class:"token string"}},[t._v('"http://data.worldbank.org/indicator/NY.GDP.MKTP.CD"')]),t._v("\n"),e("span",{pre:!0,attrs:{class:"token punctuation"}},[t._v("}")]),e("span",{pre:!0,attrs:{class:"token punctuation"}},[t._v("]")]),t._v("\n")])])]),e("ul",[e("li",[e("code",[t._v("title")]),t._v(": title of the source (e.g. document or organization name)")]),t._v(" "),e("li",[e("code",[t._v("path")]),t._v(": A "),e("RouterLink",{attrs:{to:"/data-resource/#url-or-path"}},[t._v("url-or-path")]),t._v(" string, that is a fully qualified HTTP address, or a relative POSIX path (see "),e("RouterLink",{attrs:{to:"/data-resource/#url-or-path"}},[t._v("the url-or-path definition in Data Resource for details")]),t._v(").")],1),t._v(" "),e("li",[e("code",[t._v("email")]),t._v(": An email address")])]),t._v(" "),e("h5",{attrs:{id:"contributors"}},[e("a",{staticClass:"header-anchor",attrs:{href:"#contributors"}},[t._v("#")]),t._v(" "),e("code",[t._v("contributors")])]),t._v(" "),e("p",[t._v("The people or organizations who contributed to this Data Package. It "),e("code",[t._v("MUST")]),t._v(" be an array. Each entry is a Contributor and "),e("code",[t._v("MUST")]),t._v(" be an "),e("code",[t._v("object")]),t._v(". A Contributor "),e("code",[t._v("MUST")]),t._v(" have a "),e("code",[t._v("title")]),t._v(" property and MAY contain "),e("code",[t._v("path")]),t._v(", "),e("code",[t._v("email")]),t._v(", "),e("code",[t._v("role")]),t._v(" and "),e("code",[t._v("organization")]),t._v(" properties. An example of the object structure is as follows:")]),t._v(" "),e("div",{staticClass:"language-javascript extra-class"},[e("pre",{pre:!0,attrs:{class:"language-javascript"}},[e("code",[e("span",{pre:!0,attrs:{class:"token string"}},[t._v('"contributors"')]),e("span",{pre:!0,attrs:{class:"token operator"}},[t._v(":")]),t._v(" "),e("span",{pre:!0,attrs:{class:"token punctuation"}},[t._v("[")]),e("span",{pre:!0,attrs:{class:"token punctuation"}},[t._v("{")]),t._v("\n "),e("span",{pre:!0,attrs:{class:"token string"}},[t._v('"title"')]),e("span",{pre:!0,attrs:{class:"token operator"}},[t._v(":")]),t._v(" "),e("span",{pre:!0,attrs:{class:"token string"}},[t._v('"Joe Bloggs"')]),e("span",{pre:!0,attrs:{class:"token punctuation"}},[t._v(",")]),t._v("\n "),e("span",{pre:!0,attrs:{class:"token string"}},[t._v('"email"')]),e("span",{pre:!0,attrs:{class:"token operator"}},[t._v(":")]),t._v(" "),e("span",{pre:!0,attrs:{class:"token string"}},[t._v('"joe@bloggs.com"')]),e("span",{pre:!0,attrs:{class:"token punctuation"}},[t._v(",")]),t._v("\n "),e("span",{pre:!0,attrs:{class:"token string"}},[t._v('"path"')]),e("span",{pre:!0,attrs:{class:"token operator"}},[t._v(":")]),t._v(" "),e("span",{pre:!0,attrs:{class:"token string"}},[t._v('"http://www.bloggs.com"')]),e("span",{pre:!0,attrs:{class:"token punctuation"}},[t._v(",")]),t._v("\n "),e("span",{pre:!0,attrs:{class:"token string"}},[t._v('"role"')]),e("span",{pre:!0,attrs:{class:"token operator"}},[t._v(":")]),t._v(" "),e("span",{pre:!0,attrs:{class:"token string"}},[t._v('"author"')]),t._v("\n"),e("span",{pre:!0,attrs:{class:"token punctuation"}},[t._v("}")]),e("span",{pre:!0,attrs:{class:"token punctuation"}},[t._v("]")]),t._v("\n")])])]),e("ul",[e("li",[e("code",[t._v("title")]),t._v(": name/title of the contributor (name for person, name/title of organization)")]),t._v(" "),e("li",[e("code",[t._v("path")]),t._v(": a fully qualified http URL pointing to a relevant location online for the contributor")]),t._v(" "),e("li",[e("code",[t._v("email")]),t._v(": An email address")]),t._v(" "),e("li",[e("code",[t._v("role")]),t._v(": a string describing the role of the contributor. It’s "),e("code",[t._v("RECOMMENDED")]),t._v(" to be one of: "),e("code",[t._v("author")]),t._v(", "),e("code",[t._v("publisher")]),t._v(", "),e("code",[t._v("maintainer")]),t._v(", "),e("code",[t._v("wrangler")]),t._v(", and "),e("code",[t._v("contributor")]),t._v(". Defaults to "),e("code",[t._v("contributor")]),t._v(".\n"),e("ul",[e("li",[t._v("Note on semantics: use of the “author” property does not imply that that person was the original creator of the data in the data package - merely that they created and/or maintain the data package. It is common for data packages to “package” up data from elsewhere. The original origin of the data can be indicated with the "),e("code",[t._v("sources")]),t._v(" property - see above.")])])]),t._v(" "),e("li",[e("code",[t._v("organization")]),t._v(": a string describing the organization this contributor is affiliated to.")])]),t._v(" "),e("h5",{attrs:{id:"keywords"}},[e("a",{staticClass:"header-anchor",attrs:{href:"#keywords"}},[t._v("#")]),t._v(" "),e("code",[t._v("keywords")])]),t._v(" "),e("p",[t._v("An Array of string keywords to assist users searching for the package in catalogs.")]),t._v(" "),e("h5",{attrs:{id:"image"}},[e("a",{staticClass:"header-anchor",attrs:{href:"#image"}},[t._v("#")]),t._v(" "),e("code",[t._v("image")])]),t._v(" "),e("p",[t._v("An image to use for this data package. For example, when showing the package in a listing.")]),t._v(" "),e("p",[t._v("The value of the image property "),e("code",[t._v("MUST")]),t._v(" be a string pointing to the location of the image. The string must be a "),e("RouterLink",{attrs:{to:"/data-resource/#url-or-path"}},[t._v("url-or-path")]),t._v(", that is a fully qualified HTTP address, or a relative POSIX path (see "),e("RouterLink",{attrs:{to:"/data-resource/#url-or-path"}},[t._v("the url-or-path definition in Data Resource for details")]),t._v(").")],1),t._v(" "),e("h5",{attrs:{id:"created"}},[e("a",{staticClass:"header-anchor",attrs:{href:"#created"}},[t._v("#")]),t._v(" "),e("code",[t._v("created")])]),t._v(" "),e("p",[t._v("The datetime on which this was created.")]),t._v(" "),e("p",[t._v("Note: semantics may vary between publishers – for some this is the datetime the data was created, for others the datetime the package was created.")]),t._v(" "),e("p",[t._v("The datetime must conform to the string formats for datetime as described in "),e("a",{attrs:{href:"https://tools.ietf.org/html/rfc3339#section-5.6",target:"_blank",rel:"noopener noreferrer"}},[t._v("RFC3339"),e("OutboundLink")],1),t._v(". Example:")]),t._v(" "),e("div",{staticClass:"language-javascript extra-class"},[e("pre",{pre:!0,attrs:{class:"language-javascript"}},[e("code",[e("span",{pre:!0,attrs:{class:"token punctuation"}},[t._v("{")]),t._v("\n "),e("span",{pre:!0,attrs:{class:"token string"}},[t._v('"created"')]),e("span",{pre:!0,attrs:{class:"token operator"}},[t._v(":")]),t._v(" "),e("span",{pre:!0,attrs:{class:"token string"}},[t._v('"1985-04-12T23:20:50.52Z"')]),t._v("\n"),e("span",{pre:!0,attrs:{class:"token punctuation"}},[t._v("}")]),t._v("\n")])])])],1)}),[],!1,null,null,null);a.default=r.exports}}]); \ No newline at end of file +(window.webpackJsonp=window.webpackJsonp||[]).push([[17],{386:function(t,a,e){"use strict";e.r(a);var s=e(26),r=Object(s.a)({},(function(){var t=this,a=t.$createElement,e=t._self._c||a;return e("ContentSlotsDistributor",{attrs:{"slot-key":t.$parent.slotKey}},[e("h1",{attrs:{id:"page-frontmatter-title"}},[e("a",{staticClass:"header-anchor",attrs:{href:"#page-frontmatter-title"}},[t._v("#")]),t._v(" "+t._s(t.$page.frontmatter.title))]),t._v(" "),e("p",[t._v(t._s(t.$page.frontmatter.abstract))]),t._v(" "),e("MetadataTable"),t._v(" "),e("h2",{attrs:{id:"language"}},[e("a",{staticClass:"header-anchor",attrs:{href:"#language"}},[t._v("#")]),t._v(" Language")]),t._v(" "),e("Language"),t._v(" "),e("h2",{attrs:{id:"introduction"}},[e("a",{staticClass:"header-anchor",attrs:{href:"#introduction"}},[t._v("#")]),t._v(" Introduction")]),t._v(" "),e("p",[t._v("A Data Package consists of:")]),t._v(" "),e("ul",[e("li",[t._v("Metadata that describes the structure and contents of the package")]),t._v(" "),e("li",[t._v("Resources such as data files that form the contents of the package")])]),t._v(" "),e("p",[t._v("The Data Package metadata is stored in a “descriptor”. This descriptor is what makes a collection of data a Data Package. The structure of this descriptor is the main content of the specification below.")]),t._v(" "),e("p",[t._v("In addition to this descriptor a data package will include other resources such as data files. The Data Package specification does NOT impose any requirements on their form or structure and can therefore be used for packaging "),e("strong",[t._v("any kind of data")]),t._v(".")]),t._v(" "),e("p",[t._v("The data included in the package may be provided as:")]),t._v(" "),e("ul",[e("li",[t._v("Files bundled locally with the package descriptor")]),t._v(" "),e("li",[t._v("Remote resources, referenced by URL")]),t._v(" "),e("li",[t._v("“Inline” data (see below) which is included directly in the descriptor")])]),t._v(" "),e("h3",{attrs:{id:"illustrative-structure"}},[e("a",{staticClass:"header-anchor",attrs:{href:"#illustrative-structure"}},[t._v("#")]),t._v(" Illustrative Structure")]),t._v(" "),e("p",[t._v("A minimal data package on disk would be a directory containing a single file:")]),t._v(" "),e("div",{staticClass:"language- extra-class"},[e("pre",{pre:!0,attrs:{class:"language-text"}},[e("code",[t._v("datapackage.json # (required) metadata and schemas for this data package\n")])])]),e("p",[t._v("Lacking a single external source of data would make this of limited use. A slightly less minimal version would be:")]),t._v(" "),e("div",{staticClass:"language- extra-class"},[e("pre",{pre:!0,attrs:{class:"language-text"}},[e("code",[t._v("datapackage.json\n# a data file (CSV in this case)\ndata.csv\n")])])]),e("p",[t._v("Additional files such as a README, scripts (for processing or analyzing the data) and other material may be provided. By convention scripts go in a scripts directory and thus, a more elaborate data package could look like this:")]),t._v(" "),e("div",{staticClass:"language- extra-class"},[e("pre",{pre:!0,attrs:{class:"language-text"}},[e("code",[t._v("datapackage.json # (required) metadata and schemas for this data package\nREADME.md # (optional) README in markdown format\n\n# data files may go either in data subdirectory or in main directory\nmydata.csv\ndata/otherdata.csv\n\n# the directory for code scripts - again these can go in the base directory\nscripts/my-preparation-script.py\n")])])]),e("p",[t._v("Several example data packages can be found in the "),e("a",{attrs:{href:"https://github.com/datasets",target:"_blank",rel:"noopener noreferrer"}},[t._v("datasets organization on github"),e("OutboundLink")],1),t._v(", including:")]),t._v(" "),e("ul",[e("li",[e("a",{attrs:{href:"https://github.com/datasets/gdp",target:"_blank",rel:"noopener noreferrer"}},[t._v("World GDP"),e("OutboundLink")],1)]),t._v(" "),e("li",[e("a",{attrs:{href:"https://github.com/datasets/country-codes",target:"_blank",rel:"noopener noreferrer"}},[t._v("ISO 3166-2 country codes"),e("OutboundLink")],1)])]),t._v(" "),e("h2",{attrs:{id:"specification"}},[e("a",{staticClass:"header-anchor",attrs:{href:"#specification"}},[t._v("#")]),t._v(" Specification")]),t._v(" "),e("h3",{attrs:{id:"descriptor"}},[e("a",{staticClass:"header-anchor",attrs:{href:"#descriptor"}},[t._v("#")]),t._v(" Descriptor")]),t._v(" "),e("p",[t._v("The descriptor is the central file in a Data Package. It provides:")]),t._v(" "),e("ul",[e("li",[t._v("General metadata such as the package’s title, license, publisher etc")]),t._v(" "),e("li",[t._v("A list of the data “resources” that make up the package including their location on disk or online and other relevant information (including, possibly, schema information about these data resources in a structured form)")])]),t._v(" "),e("p",[t._v("A Data Package descriptor "),e("code",[t._v("MUST")]),t._v(" be a valid JSON "),e("code",[t._v("object")]),t._v(". (JSON is defined in "),e("a",{attrs:{href:"http://www.ietf.org/rfc/rfc4627.txt",target:"_blank",rel:"noopener noreferrer"}},[t._v("RFC 4627"),e("OutboundLink")],1),t._v("). When available as a file it "),e("code",[t._v("MUST")]),t._v(" be named "),e("code",[t._v("datapackage.json")]),t._v(" and it "),e("code",[t._v("MUST")]),t._v(" be placed in the top-level directory (relative to any other resources provided as part of the data package).")]),t._v(" "),e("p",[t._v("The descriptor "),e("code",[t._v("MUST")]),t._v(" contain a "),e("code",[t._v("resources")]),t._v(" property describing the data resources.")]),t._v(" "),e("p",[t._v("All other properties are considered "),e("code",[t._v("metadata")]),t._v(" properties. The descriptor "),e("code",[t._v("MAY")]),t._v(" contain any number of other "),e("code",[t._v("metadata")]),t._v(" properties. The following sections provides a description of required and optional metadata properties for a Data Package descriptor.")]),t._v(" "),e("p",[t._v("Adherence to the specification does not imply that additional, non-specified properties cannot be used: a descriptor "),e("code",[t._v("MAY")]),t._v(" include any number of properties in additional to those described as required and optional properties. For example, if you were storing time series data and wanted to list the temporal coverage of the data in the Data Package you could add a property "),e("code",[t._v("temporal")]),t._v(" (cf "),e("a",{attrs:{href:"http://dublincore.org/documents/usageguide/qualifiers.shtml#temporal",target:"_blank",rel:"noopener noreferrer"}},[t._v("Dublin Core"),e("OutboundLink")],1),t._v("):")]),t._v(" "),e("div",{staticClass:"language-javascript extra-class"},[e("pre",{pre:!0,attrs:{class:"language-javascript"}},[e("code",[e("span",{pre:!0,attrs:{class:"token string"}},[t._v('"temporal"')]),e("span",{pre:!0,attrs:{class:"token operator"}},[t._v(":")]),t._v(" "),e("span",{pre:!0,attrs:{class:"token punctuation"}},[t._v("{")]),t._v("\n "),e("span",{pre:!0,attrs:{class:"token string"}},[t._v('"name"')]),e("span",{pre:!0,attrs:{class:"token operator"}},[t._v(":")]),t._v(" "),e("span",{pre:!0,attrs:{class:"token string"}},[t._v('"19th Century"')]),e("span",{pre:!0,attrs:{class:"token punctuation"}},[t._v(",")]),t._v("\n "),e("span",{pre:!0,attrs:{class:"token string"}},[t._v('"start"')]),e("span",{pre:!0,attrs:{class:"token operator"}},[t._v(":")]),t._v(" "),e("span",{pre:!0,attrs:{class:"token string"}},[t._v('"1800-01-01"')]),e("span",{pre:!0,attrs:{class:"token punctuation"}},[t._v(",")]),t._v("\n "),e("span",{pre:!0,attrs:{class:"token string"}},[t._v('"end"')]),e("span",{pre:!0,attrs:{class:"token operator"}},[t._v(":")]),t._v(" "),e("span",{pre:!0,attrs:{class:"token string"}},[t._v('"1899-12-31"')]),t._v("\n"),e("span",{pre:!0,attrs:{class:"token punctuation"}},[t._v("}")]),t._v("\n")])])]),e("p",[t._v("This flexibility enables specific communities to extend Data Packages as appropriate for the data they manage. As an example, the "),e("RouterLink",{attrs:{to:"/tabular-data-package/"}},[t._v("Tabular Data Package")]),t._v(" specification extends Data Package to the case where all the data is tabular and stored in CSV.")],1),t._v(" "),e("p",[t._v("Here is an illustrative example of a datapackage JSON file:")]),t._v(" "),e("div",{staticClass:"language-javascript extra-class"},[e("pre",{pre:!0,attrs:{class:"language-javascript"}},[e("code",[e("span",{pre:!0,attrs:{class:"token punctuation"}},[t._v("{")]),t._v("\n # general "),e("span",{pre:!0,attrs:{class:"token string"}},[t._v('"metadata"')]),t._v(" like title"),e("span",{pre:!0,attrs:{class:"token punctuation"}},[t._v(",")]),t._v(" sources etc\n "),e("span",{pre:!0,attrs:{class:"token string"}},[t._v('"name"')]),t._v(" "),e("span",{pre:!0,attrs:{class:"token operator"}},[t._v(":")]),t._v(" "),e("span",{pre:!0,attrs:{class:"token string"}},[t._v('"a-unique-human-readable-and-url-usable-identifier"')]),e("span",{pre:!0,attrs:{class:"token punctuation"}},[t._v(",")]),t._v("\n "),e("span",{pre:!0,attrs:{class:"token string"}},[t._v('"title"')]),t._v(" "),e("span",{pre:!0,attrs:{class:"token operator"}},[t._v(":")]),t._v(" "),e("span",{pre:!0,attrs:{class:"token string"}},[t._v('"A nice title"')]),e("span",{pre:!0,attrs:{class:"token punctuation"}},[t._v(",")]),t._v("\n "),e("span",{pre:!0,attrs:{class:"token string"}},[t._v('"licenses"')]),t._v(" "),e("span",{pre:!0,attrs:{class:"token operator"}},[t._v(":")]),t._v(" "),e("span",{pre:!0,attrs:{class:"token punctuation"}},[t._v("[")]),t._v(" "),e("span",{pre:!0,attrs:{class:"token operator"}},[t._v("...")]),t._v(" "),e("span",{pre:!0,attrs:{class:"token punctuation"}},[t._v("]")]),e("span",{pre:!0,attrs:{class:"token punctuation"}},[t._v(",")]),t._v("\n "),e("span",{pre:!0,attrs:{class:"token string"}},[t._v('"sources"')]),t._v(" "),e("span",{pre:!0,attrs:{class:"token operator"}},[t._v(":")]),t._v(" "),e("span",{pre:!0,attrs:{class:"token punctuation"}},[t._v("[")]),e("span",{pre:!0,attrs:{class:"token operator"}},[t._v("...")]),e("span",{pre:!0,attrs:{class:"token punctuation"}},[t._v("]")]),e("span",{pre:!0,attrs:{class:"token punctuation"}},[t._v(",")]),t._v("\n # list "),e("span",{pre:!0,attrs:{class:"token keyword"}},[t._v("of")]),t._v(" the data resources "),e("span",{pre:!0,attrs:{class:"token keyword"}},[t._v("in")]),t._v(" "),e("span",{pre:!0,attrs:{class:"token keyword"}},[t._v("this")]),t._v(" data "),e("span",{pre:!0,attrs:{class:"token keyword"}},[t._v("package")]),t._v("\n "),e("span",{pre:!0,attrs:{class:"token string"}},[t._v('"resources"')]),e("span",{pre:!0,attrs:{class:"token operator"}},[t._v(":")]),t._v(" "),e("span",{pre:!0,attrs:{class:"token punctuation"}},[t._v("[")]),t._v("\n "),e("span",{pre:!0,attrs:{class:"token punctuation"}},[t._v("{")]),t._v("\n "),e("span",{pre:!0,attrs:{class:"token operator"}},[t._v("...")]),t._v(" resource info described below "),e("span",{pre:!0,attrs:{class:"token operator"}},[t._v("...")]),t._v("\n "),e("span",{pre:!0,attrs:{class:"token punctuation"}},[t._v("}")]),t._v("\n "),e("span",{pre:!0,attrs:{class:"token punctuation"}},[t._v("]")]),e("span",{pre:!0,attrs:{class:"token punctuation"}},[t._v(",")]),t._v("\n # optional\n "),e("span",{pre:!0,attrs:{class:"token operator"}},[t._v("...")]),t._v(" additional information "),e("span",{pre:!0,attrs:{class:"token operator"}},[t._v("...")]),t._v("\n"),e("span",{pre:!0,attrs:{class:"token punctuation"}},[t._v("}")]),t._v("\n")])])]),e("h3",{attrs:{id:"resource-information"}},[e("a",{staticClass:"header-anchor",attrs:{href:"#resource-information"}},[t._v("#")]),t._v(" Resource Information")]),t._v(" "),e("p",[t._v("Packaged data resources are described in the "),e("code",[t._v("resources")]),t._v(" property of the package descriptor. This property "),e("code",[t._v("MUST")]),t._v(" be an array of "),e("code",[t._v("objects")]),t._v(". Each object "),e("code",[t._v("MUST")]),t._v(" follow the "),e("RouterLink",{attrs:{to:"/data-resource/"}},[t._v("Data Resource specification")]),t._v(".")],1),t._v(" "),e("h3",{attrs:{id:"metadata"}},[e("a",{staticClass:"header-anchor",attrs:{href:"#metadata"}},[t._v("#")]),t._v(" Metadata")]),t._v(" "),e("h4",{attrs:{id:"required-properties"}},[e("a",{staticClass:"header-anchor",attrs:{href:"#required-properties"}},[t._v("#")]),t._v(" Required Properties")]),t._v(" "),e("p",[t._v("The "),e("code",[t._v("resources")]),t._v(" property is required, with at least one resource.")]),t._v(" "),e("h4",{attrs:{id:"recommended-properties"}},[e("a",{staticClass:"header-anchor",attrs:{href:"#recommended-properties"}},[t._v("#")]),t._v(" Recommended Properties")]),t._v(" "),e("p",[t._v("In addition to the required properties, the following properties "),e("code",[t._v("SHOULD")]),t._v(" be included in every package descriptor:")]),t._v(" "),e("h5",{attrs:{id:"name"}},[e("a",{staticClass:"header-anchor",attrs:{href:"#name"}},[t._v("#")]),t._v(" "),e("code",[t._v("name")])]),t._v(" "),e("p",[t._v("A short url-usable (and preferably human-readable) name of the package. This "),e("code",[t._v("MUST")]),t._v(" be lower-case and contain only alphanumeric characters along with “.”, “_” or “-” characters. It will function as a unique identifier and therefore "),e("code",[t._v("SHOULD")]),t._v(" be unique in relation to any registry in which this package will be deposited (and preferably globally unique).")]),t._v(" "),e("p",[t._v("The name "),e("code",[t._v("SHOULD")]),t._v(" be invariant, meaning that it "),e("code",[t._v("SHOULD NOT")]),t._v(" change when a data package is updated, unless the new package version should be considered a distinct package, e.g. due to significant changes in structure or interpretation. Version distinction "),e("code",[t._v("SHOULD")]),t._v(" be left to the version property. As a corollary, the name also "),e("code",[t._v("SHOULD NOT")]),t._v(" include an indication of time range covered.")]),t._v(" "),e("h5",{attrs:{id:"id"}},[e("a",{staticClass:"header-anchor",attrs:{href:"#id"}},[t._v("#")]),t._v(" "),e("code",[t._v("id")])]),t._v(" "),e("p",[t._v("A property reserved for globally unique identifiers. Examples of identifiers that are unique include UUIDs and DOIs.")]),t._v(" "),e("p",[t._v("A common usage pattern for Data Packages is as a packaging format within the bounds of a system or platform. In these cases, a unique identifier for a package is desired for common data handling workflows, such as updating an existing package. While at the level of the specification, global uniqueness cannot be validated, consumers using the "),e("code",[t._v("id")]),t._v(" property "),e("code",[t._v("MUST")]),t._v(" ensure identifiers are globally unique.")]),t._v(" "),e("p",[t._v("Examples:")]),t._v(" "),e("div",{staticClass:"language-javascript extra-class"},[e("pre",{pre:!0,attrs:{class:"language-javascript"}},[e("code",[e("span",{pre:!0,attrs:{class:"token punctuation"}},[t._v("{")]),t._v("\n "),e("span",{pre:!0,attrs:{class:"token string"}},[t._v('"id"')]),e("span",{pre:!0,attrs:{class:"token operator"}},[t._v(":")]),t._v(" "),e("span",{pre:!0,attrs:{class:"token string"}},[t._v('"b03ec84-77fd-4270-813b-0c698943f7ce"')]),t._v("\n"),e("span",{pre:!0,attrs:{class:"token punctuation"}},[t._v("}")]),t._v("\n")])])]),e("div",{staticClass:"language-javascript extra-class"},[e("pre",{pre:!0,attrs:{class:"language-javascript"}},[e("code",[e("span",{pre:!0,attrs:{class:"token punctuation"}},[t._v("{")]),t._v("\n "),e("span",{pre:!0,attrs:{class:"token string"}},[t._v('"id"')]),e("span",{pre:!0,attrs:{class:"token operator"}},[t._v(":")]),t._v(" "),e("span",{pre:!0,attrs:{class:"token string"}},[t._v('"https://doi.org/10.1594/PANGAEA.726855"')]),t._v("\n"),e("span",{pre:!0,attrs:{class:"token punctuation"}},[t._v("}")]),t._v("\n")])])]),e("h5",{attrs:{id:"licenses"}},[e("a",{staticClass:"header-anchor",attrs:{href:"#licenses"}},[t._v("#")]),t._v(" "),e("code",[t._v("licenses")])]),t._v(" "),e("p",[t._v("The license(s) under which the package is provided.")]),t._v(" "),e("p",[e("strong",[t._v("This property is not legally binding and does not guarantee the package is licensed under the terms defined in this property.")])]),t._v(" "),e("p",[e("code",[t._v("licenses")]),t._v(" "),e("code",[t._v("MUST")]),t._v(" be an array. Each item in the array is a License. Each "),e("code",[t._v("MUST")]),t._v(" be an "),e("code",[t._v("object")]),t._v(". The object "),e("code",[t._v("MUST")]),t._v(" contain a "),e("code",[t._v("name")]),t._v(" property and/or a "),e("code",[t._v("path")]),t._v(" property. It "),e("code",[t._v("MAY")]),t._v(" contain a "),e("code",[t._v("title")]),t._v(" property.")]),t._v(" "),e("p",[t._v("Here is an example:")]),t._v(" "),e("div",{staticClass:"language-javascript extra-class"},[e("pre",{pre:!0,attrs:{class:"language-javascript"}},[e("code",[e("span",{pre:!0,attrs:{class:"token string"}},[t._v('"licenses"')]),e("span",{pre:!0,attrs:{class:"token operator"}},[t._v(":")]),t._v(" "),e("span",{pre:!0,attrs:{class:"token punctuation"}},[t._v("[")]),e("span",{pre:!0,attrs:{class:"token punctuation"}},[t._v("{")]),t._v("\n "),e("span",{pre:!0,attrs:{class:"token string"}},[t._v('"name"')]),e("span",{pre:!0,attrs:{class:"token operator"}},[t._v(":")]),t._v(" "),e("span",{pre:!0,attrs:{class:"token string"}},[t._v('"ODC-PDDL-1.0"')]),e("span",{pre:!0,attrs:{class:"token punctuation"}},[t._v(",")]),t._v("\n "),e("span",{pre:!0,attrs:{class:"token string"}},[t._v('"path"')]),e("span",{pre:!0,attrs:{class:"token operator"}},[t._v(":")]),t._v(" "),e("span",{pre:!0,attrs:{class:"token string"}},[t._v('"http://opendatacommons.org/licenses/pddl/"')]),e("span",{pre:!0,attrs:{class:"token punctuation"}},[t._v(",")]),t._v("\n "),e("span",{pre:!0,attrs:{class:"token string"}},[t._v('"title"')]),e("span",{pre:!0,attrs:{class:"token operator"}},[t._v(":")]),t._v(" "),e("span",{pre:!0,attrs:{class:"token string"}},[t._v('"Open Data Commons Public Domain Dedication and License v1.0"')]),t._v("\n"),e("span",{pre:!0,attrs:{class:"token punctuation"}},[t._v("}")]),e("span",{pre:!0,attrs:{class:"token punctuation"}},[t._v("]")]),t._v("\n")])])]),e("ul",[e("li",[e("code",[t._v("name")]),t._v(": The "),e("code",[t._v("name")]),t._v(" "),e("code",[t._v("MUST")]),t._v(" be an "),e("a",{attrs:{href:"http://licenses.opendefinition.org/",target:"_blank",rel:"noopener noreferrer"}},[t._v("Open Definition license ID"),e("OutboundLink")],1)]),t._v(" "),e("li",[e("code",[t._v("path")]),t._v(": A "),e("RouterLink",{attrs:{to:"/data-resource/#url-or-path"}},[t._v("url-or-path")]),t._v(" string, that is a fully qualified HTTP address, or a relative POSIX path (see "),e("RouterLink",{attrs:{to:"/data-resource/#url-or-path"}},[t._v("the url-or-path definition in Data Resource for details")]),t._v(").")],1),t._v(" "),e("li",[e("code",[t._v("title")]),t._v(": A human-readable title.")])]),t._v(" "),e("h5",{attrs:{id:"profile"}},[e("a",{staticClass:"header-anchor",attrs:{href:"#profile"}},[t._v("#")]),t._v(" "),e("code",[t._v("profile")])]),t._v(" "),e("p",[t._v("A string identifying the "),e("RouterLink",{attrs:{to:"/profiles/"}},[t._v("profile")]),t._v(" of this descriptor as per the "),e("RouterLink",{attrs:{to:"/profiles/"}},[t._v("profiles")]),t._v(" specification.")],1),t._v(" "),e("p",[t._v("Examples:")]),t._v(" "),e("div",{staticClass:"language-javascript extra-class"},[e("pre",{pre:!0,attrs:{class:"language-javascript"}},[e("code",[e("span",{pre:!0,attrs:{class:"token punctuation"}},[t._v("{")]),t._v("\n "),e("span",{pre:!0,attrs:{class:"token string"}},[t._v('"profile"')]),e("span",{pre:!0,attrs:{class:"token operator"}},[t._v(":")]),t._v(" "),e("span",{pre:!0,attrs:{class:"token string"}},[t._v('"tabular-data-package"')]),t._v("\n"),e("span",{pre:!0,attrs:{class:"token punctuation"}},[t._v("}")]),t._v("\n")])])]),e("div",{staticClass:"language-javascript extra-class"},[e("pre",{pre:!0,attrs:{class:"language-javascript"}},[e("code",[e("span",{pre:!0,attrs:{class:"token punctuation"}},[t._v("{")]),t._v("\n "),e("span",{pre:!0,attrs:{class:"token string"}},[t._v('"profile"')]),e("span",{pre:!0,attrs:{class:"token operator"}},[t._v(":")]),t._v(" "),e("span",{pre:!0,attrs:{class:"token string"}},[t._v('"http://example.com/my-profiles-json-schema.json"')]),t._v("\n"),e("span",{pre:!0,attrs:{class:"token punctuation"}},[t._v("}")]),t._v("\n")])])]),e("h4",{attrs:{id:"optional-properties"}},[e("a",{staticClass:"header-anchor",attrs:{href:"#optional-properties"}},[t._v("#")]),t._v(" Optional Properties")]),t._v(" "),e("p",[t._v("The following are commonly used properties that the package descriptor "),e("code",[t._v("MAY")]),t._v(" contain:")]),t._v(" "),e("h5",{attrs:{id:"title"}},[e("a",{staticClass:"header-anchor",attrs:{href:"#title"}},[t._v("#")]),t._v(" "),e("code",[t._v("title")])]),t._v(" "),e("p",[t._v("A "),e("code",[t._v("string")]),t._v(" providing a title or one sentence description for this package")]),t._v(" "),e("h5",{attrs:{id:"description"}},[e("a",{staticClass:"header-anchor",attrs:{href:"#description"}},[t._v("#")]),t._v(" "),e("code",[t._v("description")])]),t._v(" "),e("p",[t._v("A description of the package. The description "),e("code",[t._v("MUST")]),t._v(" be "),e("a",{attrs:{href:"http://commonmark.org/",target:"_blank",rel:"noopener noreferrer"}},[t._v("markdown"),e("OutboundLink")],1),t._v(" formatted – this also allows for simple plain text as plain text is itself valid markdown. The first paragraph (up to the first double line break) should be usable as summary information for the package.")]),t._v(" "),e("h5",{attrs:{id:"homepage"}},[e("a",{staticClass:"header-anchor",attrs:{href:"#homepage"}},[t._v("#")]),t._v(" "),e("code",[t._v("homepage")])]),t._v(" "),e("p",[t._v("A URL for the home on the web that is related to this data package.")]),t._v(" "),e("h5",{attrs:{id:"version"}},[e("a",{staticClass:"header-anchor",attrs:{href:"#version"}},[t._v("#")]),t._v(" "),e("code",[t._v("version")])]),t._v(" "),e("p",[t._v("A version string identifying the version of the package. It should conform to the "),e("a",{attrs:{href:"http://semver.org",target:"_blank",rel:"noopener noreferrer"}},[t._v("Semantic Versioning"),e("OutboundLink")],1),t._v(" requirements and should follow the "),e("RouterLink",{attrs:{to:"/patterns/#data-package-version"}},[t._v("Data Package Version")]),t._v(" pattern.")],1),t._v(" "),e("h5",{attrs:{id:"sources"}},[e("a",{staticClass:"header-anchor",attrs:{href:"#sources"}},[t._v("#")]),t._v(" "),e("code",[t._v("sources")])]),t._v(" "),e("p",[t._v("The raw sources for this data package. It "),e("code",[t._v("MUST")]),t._v(" be an array of Source objects. Each Source object "),e("code",[t._v("MUST")]),t._v(" have a "),e("code",[t._v("title")]),t._v(" and "),e("code",[t._v("MAY")]),t._v(" have "),e("code",[t._v("path")]),t._v(" and/or "),e("code",[t._v("email")]),t._v(" properties. Example:")]),t._v(" "),e("div",{staticClass:"language-javascript extra-class"},[e("pre",{pre:!0,attrs:{class:"language-javascript"}},[e("code",[e("span",{pre:!0,attrs:{class:"token string"}},[t._v('"sources"')]),e("span",{pre:!0,attrs:{class:"token operator"}},[t._v(":")]),t._v(" "),e("span",{pre:!0,attrs:{class:"token punctuation"}},[t._v("[")]),e("span",{pre:!0,attrs:{class:"token punctuation"}},[t._v("{")]),t._v("\n "),e("span",{pre:!0,attrs:{class:"token string"}},[t._v('"title"')]),e("span",{pre:!0,attrs:{class:"token operator"}},[t._v(":")]),t._v(" "),e("span",{pre:!0,attrs:{class:"token string"}},[t._v('"World Bank and OECD"')]),e("span",{pre:!0,attrs:{class:"token punctuation"}},[t._v(",")]),t._v("\n "),e("span",{pre:!0,attrs:{class:"token string"}},[t._v('"path"')]),e("span",{pre:!0,attrs:{class:"token operator"}},[t._v(":")]),t._v(" "),e("span",{pre:!0,attrs:{class:"token string"}},[t._v('"http://data.worldbank.org/indicator/NY.GDP.MKTP.CD"')]),t._v("\n"),e("span",{pre:!0,attrs:{class:"token punctuation"}},[t._v("}")]),e("span",{pre:!0,attrs:{class:"token punctuation"}},[t._v("]")]),t._v("\n")])])]),e("ul",[e("li",[e("code",[t._v("title")]),t._v(": title of the source (e.g. document or organization name)")]),t._v(" "),e("li",[e("code",[t._v("path")]),t._v(": A "),e("RouterLink",{attrs:{to:"/data-resource/#url-or-path"}},[t._v("url-or-path")]),t._v(" string, that is a fully qualified HTTP address, or a relative POSIX path (see "),e("RouterLink",{attrs:{to:"/data-resource/#url-or-path"}},[t._v("the url-or-path definition in Data Resource for details")]),t._v(").")],1),t._v(" "),e("li",[e("code",[t._v("email")]),t._v(": An email address")])]),t._v(" "),e("h5",{attrs:{id:"contributors"}},[e("a",{staticClass:"header-anchor",attrs:{href:"#contributors"}},[t._v("#")]),t._v(" "),e("code",[t._v("contributors")])]),t._v(" "),e("p",[t._v("The people or organizations who contributed to this Data Package. It "),e("code",[t._v("MUST")]),t._v(" be an array. Each entry is a Contributor and "),e("code",[t._v("MUST")]),t._v(" be an "),e("code",[t._v("object")]),t._v(". A Contributor "),e("code",[t._v("MUST")]),t._v(" have a "),e("code",[t._v("title")]),t._v(" property and MAY contain "),e("code",[t._v("path")]),t._v(", "),e("code",[t._v("email")]),t._v(", "),e("code",[t._v("role")]),t._v(" and "),e("code",[t._v("organization")]),t._v(" properties. An example of the object structure is as follows:")]),t._v(" "),e("div",{staticClass:"language-javascript extra-class"},[e("pre",{pre:!0,attrs:{class:"language-javascript"}},[e("code",[e("span",{pre:!0,attrs:{class:"token string"}},[t._v('"contributors"')]),e("span",{pre:!0,attrs:{class:"token operator"}},[t._v(":")]),t._v(" "),e("span",{pre:!0,attrs:{class:"token punctuation"}},[t._v("[")]),e("span",{pre:!0,attrs:{class:"token punctuation"}},[t._v("{")]),t._v("\n "),e("span",{pre:!0,attrs:{class:"token string"}},[t._v('"title"')]),e("span",{pre:!0,attrs:{class:"token operator"}},[t._v(":")]),t._v(" "),e("span",{pre:!0,attrs:{class:"token string"}},[t._v('"Joe Bloggs"')]),e("span",{pre:!0,attrs:{class:"token punctuation"}},[t._v(",")]),t._v("\n "),e("span",{pre:!0,attrs:{class:"token string"}},[t._v('"email"')]),e("span",{pre:!0,attrs:{class:"token operator"}},[t._v(":")]),t._v(" "),e("span",{pre:!0,attrs:{class:"token string"}},[t._v('"joe@bloggs.com"')]),e("span",{pre:!0,attrs:{class:"token punctuation"}},[t._v(",")]),t._v("\n "),e("span",{pre:!0,attrs:{class:"token string"}},[t._v('"path"')]),e("span",{pre:!0,attrs:{class:"token operator"}},[t._v(":")]),t._v(" "),e("span",{pre:!0,attrs:{class:"token string"}},[t._v('"http://www.bloggs.com"')]),e("span",{pre:!0,attrs:{class:"token punctuation"}},[t._v(",")]),t._v("\n "),e("span",{pre:!0,attrs:{class:"token string"}},[t._v('"role"')]),e("span",{pre:!0,attrs:{class:"token operator"}},[t._v(":")]),t._v(" "),e("span",{pre:!0,attrs:{class:"token string"}},[t._v('"author"')]),t._v("\n"),e("span",{pre:!0,attrs:{class:"token punctuation"}},[t._v("}")]),e("span",{pre:!0,attrs:{class:"token punctuation"}},[t._v("]")]),t._v("\n")])])]),e("ul",[e("li",[e("code",[t._v("title")]),t._v(": name/title of the contributor (name for person, name/title of organization)")]),t._v(" "),e("li",[e("code",[t._v("path")]),t._v(": a fully qualified http URL pointing to a relevant location online for the contributor")]),t._v(" "),e("li",[e("code",[t._v("email")]),t._v(": An email address")]),t._v(" "),e("li",[e("code",[t._v("role")]),t._v(": a string describing the role of the contributor. It’s "),e("code",[t._v("RECOMMENDED")]),t._v(" to be one of: "),e("code",[t._v("author")]),t._v(", "),e("code",[t._v("publisher")]),t._v(", "),e("code",[t._v("maintainer")]),t._v(", "),e("code",[t._v("wrangler")]),t._v(", and "),e("code",[t._v("contributor")]),t._v(". Defaults to "),e("code",[t._v("contributor")]),t._v(".\n"),e("ul",[e("li",[t._v("Note on semantics: use of the “author” property does not imply that that person was the original creator of the data in the data package - merely that they created and/or maintain the data package. It is common for data packages to “package” up data from elsewhere. The original origin of the data can be indicated with the "),e("code",[t._v("sources")]),t._v(" property - see above.")])])]),t._v(" "),e("li",[e("code",[t._v("organization")]),t._v(": a string describing the organization this contributor is affiliated to.")])]),t._v(" "),e("h5",{attrs:{id:"keywords"}},[e("a",{staticClass:"header-anchor",attrs:{href:"#keywords"}},[t._v("#")]),t._v(" "),e("code",[t._v("keywords")])]),t._v(" "),e("p",[t._v("An Array of string keywords to assist users searching for the package in catalogs.")]),t._v(" "),e("h5",{attrs:{id:"image"}},[e("a",{staticClass:"header-anchor",attrs:{href:"#image"}},[t._v("#")]),t._v(" "),e("code",[t._v("image")])]),t._v(" "),e("p",[t._v("An image to use for this data package. For example, when showing the package in a listing.")]),t._v(" "),e("p",[t._v("The value of the image property "),e("code",[t._v("MUST")]),t._v(" be a string pointing to the location of the image. The string must be a "),e("RouterLink",{attrs:{to:"/data-resource/#url-or-path"}},[t._v("url-or-path")]),t._v(", that is a fully qualified HTTP address, or a relative POSIX path (see "),e("RouterLink",{attrs:{to:"/data-resource/#url-or-path"}},[t._v("the url-or-path definition in Data Resource for details")]),t._v(").")],1),t._v(" "),e("h5",{attrs:{id:"created"}},[e("a",{staticClass:"header-anchor",attrs:{href:"#created"}},[t._v("#")]),t._v(" "),e("code",[t._v("created")])]),t._v(" "),e("p",[t._v("The datetime on which this was created.")]),t._v(" "),e("p",[t._v("Note: semantics may vary between publishers – for some this is the datetime the data was created, for others the datetime the package was created.")]),t._v(" "),e("p",[t._v("The datetime must conform to the string formats for datetime as described in "),e("a",{attrs:{href:"https://tools.ietf.org/html/rfc3339#section-5.6",target:"_blank",rel:"noopener noreferrer"}},[t._v("RFC3339"),e("OutboundLink")],1),t._v(". Example:")]),t._v(" "),e("div",{staticClass:"language-javascript extra-class"},[e("pre",{pre:!0,attrs:{class:"language-javascript"}},[e("code",[e("span",{pre:!0,attrs:{class:"token punctuation"}},[t._v("{")]),t._v("\n "),e("span",{pre:!0,attrs:{class:"token string"}},[t._v('"created"')]),e("span",{pre:!0,attrs:{class:"token operator"}},[t._v(":")]),t._v(" "),e("span",{pre:!0,attrs:{class:"token string"}},[t._v('"1985-04-12T23:20:50.52Z"')]),t._v("\n"),e("span",{pre:!0,attrs:{class:"token punctuation"}},[t._v("}")]),t._v("\n")])])])],1)}),[],!1,null,null,null);a.default=r.exports}}]); \ No newline at end of file diff --git a/assets/js/18.887bc200.js b/assets/js/18.503086c6.js similarity index 99% rename from assets/js/18.887bc200.js rename to assets/js/18.503086c6.js index 84ba736d..a09fba39 100644 --- a/assets/js/18.887bc200.js +++ b/assets/js/18.503086c6.js @@ -1 +1 @@ -(window.webpackJsonp=window.webpackJsonp||[]).push([[18],{386:function(t,e,a){"use strict";a.r(e);var s=a(26),r=Object(s.a)({},(function(){var t=this,e=t.$createElement,a=t._self._c||e;return a("ContentSlotsDistributor",{attrs:{"slot-key":t.$parent.slotKey}},[a("h1",{attrs:{id:"page-frontmatter-title"}},[a("a",{staticClass:"header-anchor",attrs:{href:"#page-frontmatter-title"}},[t._v("#")]),t._v(" "+t._s(t.$page.frontmatter.title))]),t._v(" "),a("p",[t._v(t._s(t.$page.frontmatter.abstract))]),t._v(" "),a("MetadataTable"),t._v(" "),a("h2",{attrs:{id:"language"}},[a("a",{staticClass:"header-anchor",attrs:{href:"#language"}},[t._v("#")]),t._v(" Language")]),t._v(" "),a("Language"),t._v(" "),a("h2",{attrs:{id:"introduction"}},[a("a",{staticClass:"header-anchor",attrs:{href:"#introduction"}},[t._v("#")]),t._v(" Introduction")]),t._v(" "),a("p",[t._v("The "),a("strong",[t._v("Data Resource")]),t._v(" format describes a data resource such as an individual file or table."),a("br"),t._v("\nThe essence of a Data Resource is a locator for the data it describes."),a("br"),t._v("\nA range of other properties can be declared to provide a richer set of metadata.")]),t._v(" "),a("h3",{attrs:{id:"examples"}},[a("a",{staticClass:"header-anchor",attrs:{href:"#examples"}},[t._v("#")]),t._v(" Examples")]),t._v(" "),a("p",[t._v("A minimal Data Resource looks as follows:")]),t._v(" "),a("p",[t._v("With data accessible via the local filesystem.")]),t._v(" "),a("div",{staticClass:"language-javascript extra-class"},[a("pre",{pre:!0,attrs:{class:"language-javascript"}},[a("code",[a("span",{pre:!0,attrs:{class:"token punctuation"}},[t._v("{")]),t._v("\n "),a("span",{pre:!0,attrs:{class:"token string"}},[t._v('"name"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[t._v(":")]),t._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[t._v('"resource-name"')]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[t._v(",")]),t._v("\n "),a("span",{pre:!0,attrs:{class:"token string"}},[t._v('"path"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[t._v(":")]),t._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[t._v('"resource-path.csv"')]),t._v("\n"),a("span",{pre:!0,attrs:{class:"token punctuation"}},[t._v("}")]),t._v("\n")])])]),a("p",[t._v("With data accessible via http.")]),t._v(" "),a("div",{staticClass:"language-javascript extra-class"},[a("pre",{pre:!0,attrs:{class:"language-javascript"}},[a("code",[a("span",{pre:!0,attrs:{class:"token punctuation"}},[t._v("{")]),t._v("\n "),a("span",{pre:!0,attrs:{class:"token string"}},[t._v('"name"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[t._v(":")]),t._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[t._v('"resource-name"')]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[t._v(",")]),t._v("\n "),a("span",{pre:!0,attrs:{class:"token string"}},[t._v('"path"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[t._v(":")]),t._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[t._v('"http://example.com/resource-path.csv"')]),t._v("\n"),a("span",{pre:!0,attrs:{class:"token punctuation"}},[t._v("}")]),t._v("\n")])])]),a("p",[t._v("A minimal Data Resource pointing to some inline data looks as follows.")]),t._v(" "),a("div",{staticClass:"language-javascript extra-class"},[a("pre",{pre:!0,attrs:{class:"language-javascript"}},[a("code",[a("span",{pre:!0,attrs:{class:"token punctuation"}},[t._v("{")]),t._v("\n "),a("span",{pre:!0,attrs:{class:"token string"}},[t._v('"name"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[t._v(":")]),t._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[t._v('"resource-name"')]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[t._v(",")]),t._v("\n "),a("span",{pre:!0,attrs:{class:"token string"}},[t._v('"data"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[t._v(":")]),t._v(" "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[t._v("{")]),t._v("\n "),a("span",{pre:!0,attrs:{class:"token string"}},[t._v('"resource-name-data"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[t._v(":")]),t._v(" "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[t._v("[")]),t._v("\n "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[t._v("{")]),a("span",{pre:!0,attrs:{class:"token string"}},[t._v('"a"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[t._v(":")]),t._v(" "),a("span",{pre:!0,attrs:{class:"token number"}},[t._v("1")]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[t._v(",")]),t._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[t._v('"b"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[t._v(":")]),t._v(" "),a("span",{pre:!0,attrs:{class:"token number"}},[t._v("2")]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[t._v("}")]),t._v("\n "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[t._v("]")]),t._v("\n "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[t._v("}")]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[t._v(",")]),t._v("\n"),a("span",{pre:!0,attrs:{class:"token punctuation"}},[t._v("}")]),t._v("\n")])])]),a("p",[t._v("A comprehensive Data Resource example with all required, recommended and optional properties looks as follows.")]),t._v(" "),a("div",{staticClass:"language-javascript extra-class"},[a("pre",{pre:!0,attrs:{class:"language-javascript"}},[a("code",[a("span",{pre:!0,attrs:{class:"token punctuation"}},[t._v("{")]),t._v("\n "),a("span",{pre:!0,attrs:{class:"token string"}},[t._v('"name"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[t._v(":")]),t._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[t._v('"solar-system"')]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[t._v(",")]),t._v("\n "),a("span",{pre:!0,attrs:{class:"token string"}},[t._v('"path"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[t._v(":")]),t._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[t._v('"http://example.com/solar-system.csv"')]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[t._v(",")]),t._v("\n "),a("span",{pre:!0,attrs:{class:"token string"}},[t._v('"title"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[t._v(":")]),t._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[t._v('"The Solar System"')]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[t._v(",")]),t._v("\n "),a("span",{pre:!0,attrs:{class:"token string"}},[t._v('"description"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[t._v(":")]),t._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[t._v('"My favourite data about the solar system."')]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[t._v(",")]),t._v("\n "),a("span",{pre:!0,attrs:{class:"token string"}},[t._v('"format"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[t._v(":")]),t._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[t._v('"csv"')]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[t._v(",")]),t._v("\n "),a("span",{pre:!0,attrs:{class:"token string"}},[t._v('"mediatype"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[t._v(":")]),t._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[t._v('"text/csv"')]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[t._v(",")]),t._v("\n "),a("span",{pre:!0,attrs:{class:"token string"}},[t._v('"encoding"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[t._v(":")]),t._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[t._v('"utf-8"')]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[t._v(",")]),t._v("\n "),a("span",{pre:!0,attrs:{class:"token string"}},[t._v('"bytes"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[t._v(":")]),t._v(" "),a("span",{pre:!0,attrs:{class:"token number"}},[t._v("1")]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[t._v(",")]),t._v("\n "),a("span",{pre:!0,attrs:{class:"token string"}},[t._v('"hash"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[t._v(":")]),t._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[t._v('""')]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[t._v(",")]),t._v("\n "),a("span",{pre:!0,attrs:{class:"token string"}},[t._v('"schema"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[t._v(":")]),t._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[t._v('""')]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[t._v(",")]),t._v("\n "),a("span",{pre:!0,attrs:{class:"token string"}},[t._v('"sources"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[t._v(":")]),t._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[t._v('""')]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[t._v(",")]),t._v("\n "),a("span",{pre:!0,attrs:{class:"token string"}},[t._v('"licenses"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[t._v(":")]),t._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[t._v('""')]),t._v("\n"),a("span",{pre:!0,attrs:{class:"token punctuation"}},[t._v("}")]),t._v("\n")])])]),a("h3",{attrs:{id:"descriptor"}},[a("a",{staticClass:"header-anchor",attrs:{href:"#descriptor"}},[t._v("#")]),t._v(" Descriptor")]),t._v(" "),a("p",[t._v("A Data Resource descriptor MUST be a valid JSON "),a("code",[t._v("object")]),t._v(". (JSON is defined in "),a("a",{attrs:{href:"http://www.ietf.org/rfc/rfc4627.txt",target:"_blank",rel:"noopener noreferrer"}},[t._v("RFC 4627"),a("OutboundLink")],1),t._v(").")]),t._v(" "),a("p",[t._v("Key properties of the descriptor are described below. A descriptor MAY include any number of properties in additional to those described below as required and optional properties.")]),t._v(" "),a("h3",{attrs:{id:"data-location"}},[a("a",{staticClass:"header-anchor",attrs:{href:"#data-location"}},[t._v("#")]),t._v(" Data Location")]),t._v(" "),a("p",[t._v("A resource MUST contain a property describing the location of the"),a("br"),t._v("\ndata associated to the resource. The location of resource data MUST be"),a("br"),t._v("\nspecified by the presence of one (and only one) of these two properties:")]),t._v(" "),a("ul",[a("li",[a("code",[t._v("path")]),t._v(": for data in files located online or locally on disk.")]),t._v(" "),a("li",[a("code",[t._v("data")]),t._v(": for data inline in the descriptor itself.")])]),t._v(" "),a("h4",{attrs:{id:"path-data-in-files"}},[a("a",{staticClass:"header-anchor",attrs:{href:"#path-data-in-files"}},[t._v("#")]),t._v(" "),a("code",[t._v("path")]),t._v(" Data in Files")]),t._v(" "),a("p",[a("code",[t._v("path")]),t._v(" MUST be a string – or an array of strings (see “Data in Multiple"),a("br"),t._v("\nFiles”). Each string MUST be a “url-or-path” as defined in the next section.")]),t._v(" "),a("h5",{attrs:{id:"url-or-path"}},[a("a",{staticClass:"header-anchor",attrs:{href:"#url-or-path"}},[t._v("#")]),t._v(" URL or Path")]),t._v(" "),a("p",[t._v("A “url-or-path” is a "),a("code",[t._v("string")]),t._v(" with the following additional constraints:")]),t._v(" "),a("ul",[a("li",[a("code",[t._v("MUST")]),t._v(" either be a URL or a POSIX path")]),t._v(" "),a("li",[a("a",{attrs:{href:"https://en.wikipedia.org/wiki/Uniform_Resource_Locator",target:"_blank",rel:"noopener noreferrer"}},[t._v("URLs"),a("OutboundLink")],1),t._v(" MUST be fully qualified. MUST be using either http or https scheme. (Absence of a scheme indicates "),a("code",[t._v("MUST")]),t._v(" be a POSIX path)")]),t._v(" "),a("li",[a("a",{attrs:{href:"https://en.wikipedia.org/wiki/Path_%28computing%29#POSIX_pathname_definition",target:"_blank",rel:"noopener noreferrer"}},[t._v("POSIX paths"),a("OutboundLink")],1),t._v(" (unix-style with "),a("code",[t._v("/")]),t._v(" as separator) are supported for referencing local files, with the security restraint that they "),a("code",[t._v("MUST")]),t._v(" be relative siblings or children of the descriptor. Absolute paths (/) and relative parent paths (…/) MUST NOT be used, and implementations SHOULD NOT support these path types.")])]),t._v(" "),a("p",[t._v("Examples:")]),t._v(" "),a("div",{staticClass:"language- extra-class"},[a("pre",{pre:!0,attrs:{class:"language-text"}},[a("code",[t._v('# fully qualified url\n"path": "http://ex.datapackages.org/big-csv/my-big.csv"\n\n# relative path\n# note: this will work both as a relative path on disk and on online\n"path": "my-data-directory/my-csv.csv"\n')])])]),a("div",{staticClass:"custom-block warning"},[a("p",{staticClass:"custom-block-title"},[t._v("WARNING")]),t._v(" "),a("p",[a("code",[t._v("/")]),t._v(" (absolute path) and "),a("code",[t._v("../")]),t._v(" (relative parent path) are forbidden to avoid security vulnerabilities when implementing data package software. These limitations on resource "),a("code",[t._v("path")]),t._v(" ensure that resource paths only point to files within the data package directory and its subdirectories. This prevents data package software being exploited by a malicious user to gain unintended access to sensitive information.")]),t._v(" "),a("p",[t._v("For example, suppose a data package hosting service stores packages on disk and allows access via an API. A malicious user uploads a data package with a resource path like "),a("code",[t._v("/etc/passwd")]),t._v(". The user then requests the data for that resource and the server naively opens "),a("code",[t._v("/etc/passwd")]),t._v(" and returns that data to the caller.")]),t._v(" "),a("p",[t._v("Prior to release 1.0.0-beta.18 (Nov 17 2016) there was a "),a("code",[t._v("url")]),t._v(" property distinct from "),a("code",[t._v("path")]),t._v(". In order to support backwards compatibility, implementors MAY want to automatically convert a "),a("code",[t._v("url")]),t._v(" property to a "),a("code",[t._v("path")]),t._v(" property and issue a warning.")])]),t._v(" "),a("h4",{attrs:{id:"data-in-multiple-files"}},[a("a",{staticClass:"header-anchor",attrs:{href:"#data-in-multiple-files"}},[t._v("#")]),t._v(" Data in Multiple Files")]),t._v(" "),a("p",[t._v("Usually, a resource will have only a single file associated to it. However, sometimes it may be convenient to have a single resource whose data is split across multiple files – perhaps the data is large and having it in one file would be inconvenient.")]),t._v(" "),a("p",[t._v("To support this use case the "),a("code",[t._v("path")]),t._v(" property MAY be an array of strings rather"),a("br"),t._v("\nthan a single string:")]),t._v(" "),a("div",{staticClass:"language- extra-class"},[a("pre",{pre:!0,attrs:{class:"language-text"}},[a("code",[t._v('"path": [ "myfile1.csv", "myfile2.csv" ]\n')])])]),a("p",[t._v("It is NOT permitted to mix fully qualified URLs and relative paths in a "),a("code",[t._v("path")]),t._v(" array: strings MUST either all be relative paths or all URLs.")]),t._v(" "),a("p",[a("strong",[t._v("NOTE:")]),t._v(" All files in the array MUST be similar in terms of structure, format etc. Implementors MUST be able to concatenate together the files in the simplest way and treat the result as one large file. For tabular data there is the issue of header rows. See the "),a("RouterLink",{attrs:{to:"/tabular-data-package/"}},[t._v("Tabular Data Package spec")]),t._v(" for more on this.")],1),t._v(" "),a("h4",{attrs:{id:"data-inline-data"}},[a("a",{staticClass:"header-anchor",attrs:{href:"#data-inline-data"}},[t._v("#")]),t._v(" "),a("code",[t._v("data")]),t._v(" Inline Data")]),t._v(" "),a("p",[t._v("Resource data rather than being stored in external files can be shipped "),a("code",[t._v("inline")]),t._v(" on a Resource using the "),a("code",[t._v("data")]),t._v(" property.")]),t._v(" "),a("p",[t._v("The value of the data property can be any type of data. However, restrictions of JSON require that the value be a string so for binary data you will need to encode (e.g. to Base64). Information on the type and encoding of the value of the data property SHOULD be provided by the format (or mediatype) property and the encoding property.")]),t._v(" "),a("p",[t._v("Specifically: the value of the data property MUST be:")]),t._v(" "),a("ul",[a("li",[t._v("EITHER: a "),a("strong",[t._v("JSON")]),t._v(" array or "),a("strong",[t._v("Object")]),t._v("- the data is then assumed to be JSON data and SHOULD be processed as such")]),t._v(" "),a("li",[t._v("OR: a "),a("strong",[t._v("JSON")]),t._v(" string - in this case the format or mediatype properties MUST be provided.")])]),t._v(" "),a("p",[t._v("Thus, a consumer of resource object MAY assume if no format or mediatype property is provided that the data is JSON and attempt to process it as such.")]),t._v(" "),a("p",[a("strong",[t._v("Examples 1 - inline JSON:")])]),t._v(" "),a("div",{staticClass:"language- extra-class"},[a("pre",[a("code",[t._v('{\n ...\n "resources": [\n {\n "format": "json",\n # some json data e.g.\n "data": [\n { "a": 1, "b": 2 },\n { .... }\n ]\n }\n ]\n}\n')])])]),a("p",[a("strong",[t._v("Example 2 - inline CSV:")])]),t._v(" "),a("div",{staticClass:"language- extra-class"},[a("pre",[a("code",[t._v('{\n ...\n "resources": [\n {\n "format": "csv",\n "data": "A,B,C\\n1,2,3\\n4,5,6"\n }\n ]\n}\n')])])]),a("h3",{attrs:{id:"metadata-properties"}},[a("a",{staticClass:"header-anchor",attrs:{href:"#metadata-properties"}},[t._v("#")]),t._v(" Metadata Properties")]),t._v(" "),a("h4",{attrs:{id:"required-properties"}},[a("a",{staticClass:"header-anchor",attrs:{href:"#required-properties"}},[t._v("#")]),t._v(" Required Properties")]),t._v(" "),a("p",[t._v("A descriptor MUST contain the following properties:")]),t._v(" "),a("h4",{attrs:{id:"name"}},[a("a",{staticClass:"header-anchor",attrs:{href:"#name"}},[t._v("#")]),t._v(" "),a("code",[t._v("name")])]),t._v(" "),a("p",[t._v("A resource MUST contain a "),a("code",[t._v("name")]),t._v(" property. The name is a simple name or"),a("br"),t._v("\nidentifier to be used for this resource.")]),t._v(" "),a("ul",[a("li",[t._v("If present, the name MUST be unique amongst all resources in this data"),a("br"),t._v("\npackage.")]),t._v(" "),a("li",[t._v("It MUST consist only of lowercase alphanumeric characters plus “.”, “-” and “_”.")]),t._v(" "),a("li",[t._v("It would be usual for the name to correspond to the file name (minus the"),a("br"),t._v("\nextension) of the data file the resource describes.")])]),t._v(" "),a("h4",{attrs:{id:"recommended-properties"}},[a("a",{staticClass:"header-anchor",attrs:{href:"#recommended-properties"}},[t._v("#")]),t._v(" Recommended Properties")]),t._v(" "),a("h4",{attrs:{id:"profile"}},[a("a",{staticClass:"header-anchor",attrs:{href:"#profile"}},[t._v("#")]),t._v(" "),a("code",[t._v("profile")])]),t._v(" "),a("p",[t._v("A string identifying the "),a("RouterLink",{attrs:{to:"/profiles/"}},[t._v("profile")]),t._v(" of this descriptor as per the "),a("RouterLink",{attrs:{to:"/profiles/"}},[t._v("profiles")]),t._v(" specification.")],1),t._v(" "),a("p",[t._v("Examples:")]),t._v(" "),a("div",{staticClass:"language-javascript extra-class"},[a("pre",{pre:!0,attrs:{class:"language-javascript"}},[a("code",[a("span",{pre:!0,attrs:{class:"token punctuation"}},[t._v("{")]),t._v("\n "),a("span",{pre:!0,attrs:{class:"token string"}},[t._v('"profile"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[t._v(":")]),t._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[t._v('"tabular-data-resource"')]),t._v("\n"),a("span",{pre:!0,attrs:{class:"token punctuation"}},[t._v("}")]),t._v("\n")])])]),a("div",{staticClass:"language- extra-class"},[a("pre",{pre:!0,attrs:{class:"language-text"}},[a("code",[t._v('{\n "profile": "http://example.com/my-profiles-json-schema.json"\n}\n')])])]),a("h4",{attrs:{id:"optional-properties"}},[a("a",{staticClass:"header-anchor",attrs:{href:"#optional-properties"}},[t._v("#")]),t._v(" Optional Properties")]),t._v(" "),a("p",[t._v("A descriptor MAY contain any number of additional properties. Common properties include:")]),t._v(" "),a("ul",[a("li",[a("p",[a("code",[t._v("title")]),t._v(": a title or label for the resource.")])]),t._v(" "),a("li",[a("p",[a("code",[t._v("description")]),t._v(": a description of the resource.")])]),t._v(" "),a("li",[a("p",[a("code",[t._v("format")]),t._v(": ‘csv’, ‘xls’, ‘json’ etc. Would be expected to be the standard file"),a("br"),t._v("\nextension for this type of resource.")])]),t._v(" "),a("li",[a("p",[a("code",[t._v("mediatype")]),t._v(": the mediatype/mimetype of the resource e.g. “text/csv”, or “application/vnd.ms-excel”. Mediatypes are maintained by the Internet Assigned Numbers Authority (IANA) in a "),a("a",{attrs:{href:"https://www.iana.org/assignments/media-types/media-types.xhtml",target:"_blank",rel:"noopener noreferrer"}},[t._v("media type registry"),a("OutboundLink")],1),t._v(".")])]),t._v(" "),a("li",[a("p",[a("code",[t._v("encoding")]),t._v(": specify the character encoding of the resource’s data file. The values should be one of"),a("br"),t._v("\nthe “Preferred MIME Names” for "),a("a",{attrs:{href:"http://www.iana.org/assignments/character-sets/character-sets.xhtml",target:"_blank",rel:"noopener noreferrer"}},[t._v("a character encoding registered with IANA"),a("OutboundLink")],1),t._v(". If no"),a("br"),t._v("\nvalue for this key is specified then the default is UTF-8.")])]),t._v(" "),a("li",[a("p",[a("code",[t._v("bytes")]),t._v(": size of the file in bytes.")])]),t._v(" "),a("li",[a("p",[a("code",[t._v("hash")]),t._v(": the MD5 hash for this resource. Other algorithms can be indicated by prefixing"),a("br"),t._v("\nthe hash’s value with the algorithm name in lower-case. For example:")]),t._v(" "),a("div",{staticClass:"language- extra-class"},[a("pre",[a("code",[t._v('"hash": "sha1:8843d7f92416211de9ebb963ff4ce28125932878"\n')])])])]),t._v(" "),a("li",[a("p",[a("code",[t._v("sources")]),t._v(": as for "),a("RouterLink",{attrs:{to:"/data-package/"}},[t._v("Data Package metadata")]),t._v(".")],1)]),t._v(" "),a("li",[a("p",[a("code",[t._v("licenses")]),t._v(": as for "),a("RouterLink",{attrs:{to:"/data-package/"}},[t._v("Data Package metadata")]),t._v(". If not specified the resource"),a("br"),t._v("\ninherits from the data package.")],1)])]),t._v(" "),a("h3",{attrs:{id:"resource-schemas"}},[a("a",{staticClass:"header-anchor",attrs:{href:"#resource-schemas"}},[t._v("#")]),t._v(" Resource Schemas")]),t._v(" "),a("p",[t._v("A Data Resource MAY have a "),a("code",[t._v("schema")]),t._v(" property to describe the schema of the resource data.")]),t._v(" "),a("p",[t._v("The value for the "),a("code",[t._v("schema")]),t._v(" property on a "),a("code",[t._v("resource")]),t._v(" MUST be an "),a("code",[t._v("object")]),t._v(" representing the schema OR a "),a("code",[t._v("string")]),t._v(" that identifies the location of the schema.")]),t._v(" "),a("p",[t._v("If a "),a("code",[t._v("string")]),t._v(" it must be a "),a("a",{attrs:{href:"#url-or-path"}},[t._v("url-or-path as defined above")]),t._v(", that is a fully qualified http URL or a relative POSIX path. The file at the location specified by this url-or-path string "),a("code",[t._v("MUST")]),t._v(" be a JSON document containing the schema.")]),t._v(" "),a("p",[t._v("NOTE: the Data Package specification places no restrictions on the form of the schema Object. This flexibility enables specific communities to define schemas appropriate for the data they manage. As an example, the "),a("RouterLink",{attrs:{to:"/tabular-data-package/"}},[t._v("Tabular Data Package")]),t._v(" specification requires the schema to conform to "),a("RouterLink",{attrs:{to:"/table-schema/"}},[t._v("Table Schema")]),t._v(".")],1)],1)}),[],!1,null,null,null);e.default=r.exports}}]); \ No newline at end of file +(window.webpackJsonp=window.webpackJsonp||[]).push([[18],{387:function(t,e,a){"use strict";a.r(e);var s=a(26),r=Object(s.a)({},(function(){var t=this,e=t.$createElement,a=t._self._c||e;return a("ContentSlotsDistributor",{attrs:{"slot-key":t.$parent.slotKey}},[a("h1",{attrs:{id:"page-frontmatter-title"}},[a("a",{staticClass:"header-anchor",attrs:{href:"#page-frontmatter-title"}},[t._v("#")]),t._v(" "+t._s(t.$page.frontmatter.title))]),t._v(" "),a("p",[t._v(t._s(t.$page.frontmatter.abstract))]),t._v(" "),a("MetadataTable"),t._v(" "),a("h2",{attrs:{id:"language"}},[a("a",{staticClass:"header-anchor",attrs:{href:"#language"}},[t._v("#")]),t._v(" Language")]),t._v(" "),a("Language"),t._v(" "),a("h2",{attrs:{id:"introduction"}},[a("a",{staticClass:"header-anchor",attrs:{href:"#introduction"}},[t._v("#")]),t._v(" Introduction")]),t._v(" "),a("p",[t._v("The "),a("strong",[t._v("Data Resource")]),t._v(" format describes a data resource such as an individual file or table."),a("br"),t._v("\nThe essence of a Data Resource is a locator for the data it describes."),a("br"),t._v("\nA range of other properties can be declared to provide a richer set of metadata.")]),t._v(" "),a("h3",{attrs:{id:"examples"}},[a("a",{staticClass:"header-anchor",attrs:{href:"#examples"}},[t._v("#")]),t._v(" Examples")]),t._v(" "),a("p",[t._v("A minimal Data Resource looks as follows:")]),t._v(" "),a("p",[t._v("With data accessible via the local filesystem.")]),t._v(" "),a("div",{staticClass:"language-javascript extra-class"},[a("pre",{pre:!0,attrs:{class:"language-javascript"}},[a("code",[a("span",{pre:!0,attrs:{class:"token punctuation"}},[t._v("{")]),t._v("\n "),a("span",{pre:!0,attrs:{class:"token string"}},[t._v('"name"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[t._v(":")]),t._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[t._v('"resource-name"')]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[t._v(",")]),t._v("\n "),a("span",{pre:!0,attrs:{class:"token string"}},[t._v('"path"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[t._v(":")]),t._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[t._v('"resource-path.csv"')]),t._v("\n"),a("span",{pre:!0,attrs:{class:"token punctuation"}},[t._v("}")]),t._v("\n")])])]),a("p",[t._v("With data accessible via http.")]),t._v(" "),a("div",{staticClass:"language-javascript extra-class"},[a("pre",{pre:!0,attrs:{class:"language-javascript"}},[a("code",[a("span",{pre:!0,attrs:{class:"token punctuation"}},[t._v("{")]),t._v("\n "),a("span",{pre:!0,attrs:{class:"token string"}},[t._v('"name"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[t._v(":")]),t._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[t._v('"resource-name"')]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[t._v(",")]),t._v("\n "),a("span",{pre:!0,attrs:{class:"token string"}},[t._v('"path"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[t._v(":")]),t._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[t._v('"http://example.com/resource-path.csv"')]),t._v("\n"),a("span",{pre:!0,attrs:{class:"token punctuation"}},[t._v("}")]),t._v("\n")])])]),a("p",[t._v("A minimal Data Resource pointing to some inline data looks as follows.")]),t._v(" "),a("div",{staticClass:"language-javascript extra-class"},[a("pre",{pre:!0,attrs:{class:"language-javascript"}},[a("code",[a("span",{pre:!0,attrs:{class:"token punctuation"}},[t._v("{")]),t._v("\n "),a("span",{pre:!0,attrs:{class:"token string"}},[t._v('"name"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[t._v(":")]),t._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[t._v('"resource-name"')]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[t._v(",")]),t._v("\n "),a("span",{pre:!0,attrs:{class:"token string"}},[t._v('"data"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[t._v(":")]),t._v(" "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[t._v("{")]),t._v("\n "),a("span",{pre:!0,attrs:{class:"token string"}},[t._v('"resource-name-data"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[t._v(":")]),t._v(" "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[t._v("[")]),t._v("\n "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[t._v("{")]),a("span",{pre:!0,attrs:{class:"token string"}},[t._v('"a"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[t._v(":")]),t._v(" "),a("span",{pre:!0,attrs:{class:"token number"}},[t._v("1")]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[t._v(",")]),t._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[t._v('"b"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[t._v(":")]),t._v(" "),a("span",{pre:!0,attrs:{class:"token number"}},[t._v("2")]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[t._v("}")]),t._v("\n "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[t._v("]")]),t._v("\n "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[t._v("}")]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[t._v(",")]),t._v("\n"),a("span",{pre:!0,attrs:{class:"token punctuation"}},[t._v("}")]),t._v("\n")])])]),a("p",[t._v("A comprehensive Data Resource example with all required, recommended and optional properties looks as follows.")]),t._v(" "),a("div",{staticClass:"language-javascript extra-class"},[a("pre",{pre:!0,attrs:{class:"language-javascript"}},[a("code",[a("span",{pre:!0,attrs:{class:"token punctuation"}},[t._v("{")]),t._v("\n "),a("span",{pre:!0,attrs:{class:"token string"}},[t._v('"name"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[t._v(":")]),t._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[t._v('"solar-system"')]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[t._v(",")]),t._v("\n "),a("span",{pre:!0,attrs:{class:"token string"}},[t._v('"path"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[t._v(":")]),t._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[t._v('"http://example.com/solar-system.csv"')]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[t._v(",")]),t._v("\n "),a("span",{pre:!0,attrs:{class:"token string"}},[t._v('"title"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[t._v(":")]),t._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[t._v('"The Solar System"')]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[t._v(",")]),t._v("\n "),a("span",{pre:!0,attrs:{class:"token string"}},[t._v('"description"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[t._v(":")]),t._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[t._v('"My favourite data about the solar system."')]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[t._v(",")]),t._v("\n "),a("span",{pre:!0,attrs:{class:"token string"}},[t._v('"format"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[t._v(":")]),t._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[t._v('"csv"')]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[t._v(",")]),t._v("\n "),a("span",{pre:!0,attrs:{class:"token string"}},[t._v('"mediatype"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[t._v(":")]),t._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[t._v('"text/csv"')]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[t._v(",")]),t._v("\n "),a("span",{pre:!0,attrs:{class:"token string"}},[t._v('"encoding"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[t._v(":")]),t._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[t._v('"utf-8"')]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[t._v(",")]),t._v("\n "),a("span",{pre:!0,attrs:{class:"token string"}},[t._v('"bytes"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[t._v(":")]),t._v(" "),a("span",{pre:!0,attrs:{class:"token number"}},[t._v("1")]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[t._v(",")]),t._v("\n "),a("span",{pre:!0,attrs:{class:"token string"}},[t._v('"hash"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[t._v(":")]),t._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[t._v('""')]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[t._v(",")]),t._v("\n "),a("span",{pre:!0,attrs:{class:"token string"}},[t._v('"schema"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[t._v(":")]),t._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[t._v('""')]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[t._v(",")]),t._v("\n "),a("span",{pre:!0,attrs:{class:"token string"}},[t._v('"sources"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[t._v(":")]),t._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[t._v('""')]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[t._v(",")]),t._v("\n "),a("span",{pre:!0,attrs:{class:"token string"}},[t._v('"licenses"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[t._v(":")]),t._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[t._v('""')]),t._v("\n"),a("span",{pre:!0,attrs:{class:"token punctuation"}},[t._v("}")]),t._v("\n")])])]),a("h3",{attrs:{id:"descriptor"}},[a("a",{staticClass:"header-anchor",attrs:{href:"#descriptor"}},[t._v("#")]),t._v(" Descriptor")]),t._v(" "),a("p",[t._v("A Data Resource descriptor MUST be a valid JSON "),a("code",[t._v("object")]),t._v(". (JSON is defined in "),a("a",{attrs:{href:"http://www.ietf.org/rfc/rfc4627.txt",target:"_blank",rel:"noopener noreferrer"}},[t._v("RFC 4627"),a("OutboundLink")],1),t._v(").")]),t._v(" "),a("p",[t._v("Key properties of the descriptor are described below. A descriptor MAY include any number of properties in additional to those described below as required and optional properties.")]),t._v(" "),a("h3",{attrs:{id:"data-location"}},[a("a",{staticClass:"header-anchor",attrs:{href:"#data-location"}},[t._v("#")]),t._v(" Data Location")]),t._v(" "),a("p",[t._v("A resource MUST contain a property describing the location of the"),a("br"),t._v("\ndata associated to the resource. The location of resource data MUST be"),a("br"),t._v("\nspecified by the presence of one (and only one) of these two properties:")]),t._v(" "),a("ul",[a("li",[a("code",[t._v("path")]),t._v(": for data in files located online or locally on disk.")]),t._v(" "),a("li",[a("code",[t._v("data")]),t._v(": for data inline in the descriptor itself.")])]),t._v(" "),a("h4",{attrs:{id:"path-data-in-files"}},[a("a",{staticClass:"header-anchor",attrs:{href:"#path-data-in-files"}},[t._v("#")]),t._v(" "),a("code",[t._v("path")]),t._v(" Data in Files")]),t._v(" "),a("p",[a("code",[t._v("path")]),t._v(" MUST be a string – or an array of strings (see “Data in Multiple"),a("br"),t._v("\nFiles”). Each string MUST be a “url-or-path” as defined in the next section.")]),t._v(" "),a("h5",{attrs:{id:"url-or-path"}},[a("a",{staticClass:"header-anchor",attrs:{href:"#url-or-path"}},[t._v("#")]),t._v(" URL or Path")]),t._v(" "),a("p",[t._v("A “url-or-path” is a "),a("code",[t._v("string")]),t._v(" with the following additional constraints:")]),t._v(" "),a("ul",[a("li",[a("code",[t._v("MUST")]),t._v(" either be a URL or a POSIX path")]),t._v(" "),a("li",[a("a",{attrs:{href:"https://en.wikipedia.org/wiki/Uniform_Resource_Locator",target:"_blank",rel:"noopener noreferrer"}},[t._v("URLs"),a("OutboundLink")],1),t._v(" MUST be fully qualified. MUST be using either http or https scheme. (Absence of a scheme indicates "),a("code",[t._v("MUST")]),t._v(" be a POSIX path)")]),t._v(" "),a("li",[a("a",{attrs:{href:"https://en.wikipedia.org/wiki/Path_%28computing%29#POSIX_pathname_definition",target:"_blank",rel:"noopener noreferrer"}},[t._v("POSIX paths"),a("OutboundLink")],1),t._v(" (unix-style with "),a("code",[t._v("/")]),t._v(" as separator) are supported for referencing local files, with the security restraint that they "),a("code",[t._v("MUST")]),t._v(" be relative siblings or children of the descriptor. Absolute paths (/) and relative parent paths (…/) MUST NOT be used, and implementations SHOULD NOT support these path types.")])]),t._v(" "),a("p",[t._v("Examples:")]),t._v(" "),a("div",{staticClass:"language- extra-class"},[a("pre",{pre:!0,attrs:{class:"language-text"}},[a("code",[t._v('# fully qualified url\n"path": "http://ex.datapackages.org/big-csv/my-big.csv"\n\n# relative path\n# note: this will work both as a relative path on disk and on online\n"path": "my-data-directory/my-csv.csv"\n')])])]),a("div",{staticClass:"custom-block warning"},[a("p",{staticClass:"custom-block-title"},[t._v("WARNING")]),t._v(" "),a("p",[a("code",[t._v("/")]),t._v(" (absolute path) and "),a("code",[t._v("../")]),t._v(" (relative parent path) are forbidden to avoid security vulnerabilities when implementing data package software. These limitations on resource "),a("code",[t._v("path")]),t._v(" ensure that resource paths only point to files within the data package directory and its subdirectories. This prevents data package software being exploited by a malicious user to gain unintended access to sensitive information.")]),t._v(" "),a("p",[t._v("For example, suppose a data package hosting service stores packages on disk and allows access via an API. A malicious user uploads a data package with a resource path like "),a("code",[t._v("/etc/passwd")]),t._v(". The user then requests the data for that resource and the server naively opens "),a("code",[t._v("/etc/passwd")]),t._v(" and returns that data to the caller.")]),t._v(" "),a("p",[t._v("Prior to release 1.0.0-beta.18 (Nov 17 2016) there was a "),a("code",[t._v("url")]),t._v(" property distinct from "),a("code",[t._v("path")]),t._v(". In order to support backwards compatibility, implementors MAY want to automatically convert a "),a("code",[t._v("url")]),t._v(" property to a "),a("code",[t._v("path")]),t._v(" property and issue a warning.")])]),t._v(" "),a("h4",{attrs:{id:"data-in-multiple-files"}},[a("a",{staticClass:"header-anchor",attrs:{href:"#data-in-multiple-files"}},[t._v("#")]),t._v(" Data in Multiple Files")]),t._v(" "),a("p",[t._v("Usually, a resource will have only a single file associated to it. However, sometimes it may be convenient to have a single resource whose data is split across multiple files – perhaps the data is large and having it in one file would be inconvenient.")]),t._v(" "),a("p",[t._v("To support this use case the "),a("code",[t._v("path")]),t._v(" property MAY be an array of strings rather"),a("br"),t._v("\nthan a single string:")]),t._v(" "),a("div",{staticClass:"language- extra-class"},[a("pre",{pre:!0,attrs:{class:"language-text"}},[a("code",[t._v('"path": [ "myfile1.csv", "myfile2.csv" ]\n')])])]),a("p",[t._v("It is NOT permitted to mix fully qualified URLs and relative paths in a "),a("code",[t._v("path")]),t._v(" array: strings MUST either all be relative paths or all URLs.")]),t._v(" "),a("p",[a("strong",[t._v("NOTE:")]),t._v(" All files in the array MUST be similar in terms of structure, format etc. Implementors MUST be able to concatenate together the files in the simplest way and treat the result as one large file. For tabular data there is the issue of header rows. See the "),a("RouterLink",{attrs:{to:"/tabular-data-package/"}},[t._v("Tabular Data Package spec")]),t._v(" for more on this.")],1),t._v(" "),a("h4",{attrs:{id:"data-inline-data"}},[a("a",{staticClass:"header-anchor",attrs:{href:"#data-inline-data"}},[t._v("#")]),t._v(" "),a("code",[t._v("data")]),t._v(" Inline Data")]),t._v(" "),a("p",[t._v("Resource data rather than being stored in external files can be shipped "),a("code",[t._v("inline")]),t._v(" on a Resource using the "),a("code",[t._v("data")]),t._v(" property.")]),t._v(" "),a("p",[t._v("The value of the data property can be any type of data. However, restrictions of JSON require that the value be a string so for binary data you will need to encode (e.g. to Base64). Information on the type and encoding of the value of the data property SHOULD be provided by the format (or mediatype) property and the encoding property.")]),t._v(" "),a("p",[t._v("Specifically: the value of the data property MUST be:")]),t._v(" "),a("ul",[a("li",[t._v("EITHER: a "),a("strong",[t._v("JSON")]),t._v(" array or "),a("strong",[t._v("Object")]),t._v("- the data is then assumed to be JSON data and SHOULD be processed as such")]),t._v(" "),a("li",[t._v("OR: a "),a("strong",[t._v("JSON")]),t._v(" string - in this case the format or mediatype properties MUST be provided.")])]),t._v(" "),a("p",[t._v("Thus, a consumer of resource object MAY assume if no format or mediatype property is provided that the data is JSON and attempt to process it as such.")]),t._v(" "),a("p",[a("strong",[t._v("Examples 1 - inline JSON:")])]),t._v(" "),a("div",{staticClass:"language- extra-class"},[a("pre",[a("code",[t._v('{\n ...\n "resources": [\n {\n "format": "json",\n # some json data e.g.\n "data": [\n { "a": 1, "b": 2 },\n { .... }\n ]\n }\n ]\n}\n')])])]),a("p",[a("strong",[t._v("Example 2 - inline CSV:")])]),t._v(" "),a("div",{staticClass:"language- extra-class"},[a("pre",[a("code",[t._v('{\n ...\n "resources": [\n {\n "format": "csv",\n "data": "A,B,C\\n1,2,3\\n4,5,6"\n }\n ]\n}\n')])])]),a("h3",{attrs:{id:"metadata-properties"}},[a("a",{staticClass:"header-anchor",attrs:{href:"#metadata-properties"}},[t._v("#")]),t._v(" Metadata Properties")]),t._v(" "),a("h4",{attrs:{id:"required-properties"}},[a("a",{staticClass:"header-anchor",attrs:{href:"#required-properties"}},[t._v("#")]),t._v(" Required Properties")]),t._v(" "),a("p",[t._v("A descriptor MUST contain the following properties:")]),t._v(" "),a("h4",{attrs:{id:"name"}},[a("a",{staticClass:"header-anchor",attrs:{href:"#name"}},[t._v("#")]),t._v(" "),a("code",[t._v("name")])]),t._v(" "),a("p",[t._v("A resource MUST contain a "),a("code",[t._v("name")]),t._v(" property. The name is a simple name or"),a("br"),t._v("\nidentifier to be used for this resource.")]),t._v(" "),a("ul",[a("li",[t._v("If present, the name MUST be unique amongst all resources in this data"),a("br"),t._v("\npackage.")]),t._v(" "),a("li",[t._v("It MUST consist only of lowercase alphanumeric characters plus “.”, “-” and “_”.")]),t._v(" "),a("li",[t._v("It would be usual for the name to correspond to the file name (minus the"),a("br"),t._v("\nextension) of the data file the resource describes.")])]),t._v(" "),a("h4",{attrs:{id:"recommended-properties"}},[a("a",{staticClass:"header-anchor",attrs:{href:"#recommended-properties"}},[t._v("#")]),t._v(" Recommended Properties")]),t._v(" "),a("h4",{attrs:{id:"profile"}},[a("a",{staticClass:"header-anchor",attrs:{href:"#profile"}},[t._v("#")]),t._v(" "),a("code",[t._v("profile")])]),t._v(" "),a("p",[t._v("A string identifying the "),a("RouterLink",{attrs:{to:"/profiles/"}},[t._v("profile")]),t._v(" of this descriptor as per the "),a("RouterLink",{attrs:{to:"/profiles/"}},[t._v("profiles")]),t._v(" specification.")],1),t._v(" "),a("p",[t._v("Examples:")]),t._v(" "),a("div",{staticClass:"language-javascript extra-class"},[a("pre",{pre:!0,attrs:{class:"language-javascript"}},[a("code",[a("span",{pre:!0,attrs:{class:"token punctuation"}},[t._v("{")]),t._v("\n "),a("span",{pre:!0,attrs:{class:"token string"}},[t._v('"profile"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[t._v(":")]),t._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[t._v('"tabular-data-resource"')]),t._v("\n"),a("span",{pre:!0,attrs:{class:"token punctuation"}},[t._v("}")]),t._v("\n")])])]),a("div",{staticClass:"language- extra-class"},[a("pre",{pre:!0,attrs:{class:"language-text"}},[a("code",[t._v('{\n "profile": "http://example.com/my-profiles-json-schema.json"\n}\n')])])]),a("h4",{attrs:{id:"optional-properties"}},[a("a",{staticClass:"header-anchor",attrs:{href:"#optional-properties"}},[t._v("#")]),t._v(" Optional Properties")]),t._v(" "),a("p",[t._v("A descriptor MAY contain any number of additional properties. Common properties include:")]),t._v(" "),a("ul",[a("li",[a("p",[a("code",[t._v("title")]),t._v(": a title or label for the resource.")])]),t._v(" "),a("li",[a("p",[a("code",[t._v("description")]),t._v(": a description of the resource.")])]),t._v(" "),a("li",[a("p",[a("code",[t._v("format")]),t._v(": ‘csv’, ‘xls’, ‘json’ etc. Would be expected to be the standard file"),a("br"),t._v("\nextension for this type of resource.")])]),t._v(" "),a("li",[a("p",[a("code",[t._v("mediatype")]),t._v(": the mediatype/mimetype of the resource e.g. “text/csv”, or “application/vnd.ms-excel”. Mediatypes are maintained by the Internet Assigned Numbers Authority (IANA) in a "),a("a",{attrs:{href:"https://www.iana.org/assignments/media-types/media-types.xhtml",target:"_blank",rel:"noopener noreferrer"}},[t._v("media type registry"),a("OutboundLink")],1),t._v(".")])]),t._v(" "),a("li",[a("p",[a("code",[t._v("encoding")]),t._v(": specify the character encoding of the resource’s data file. The values should be one of"),a("br"),t._v("\nthe “Preferred MIME Names” for "),a("a",{attrs:{href:"http://www.iana.org/assignments/character-sets/character-sets.xhtml",target:"_blank",rel:"noopener noreferrer"}},[t._v("a character encoding registered with IANA"),a("OutboundLink")],1),t._v(". If no"),a("br"),t._v("\nvalue for this key is specified then the default is UTF-8.")])]),t._v(" "),a("li",[a("p",[a("code",[t._v("bytes")]),t._v(": size of the file in bytes.")])]),t._v(" "),a("li",[a("p",[a("code",[t._v("hash")]),t._v(": the MD5 hash for this resource. Other algorithms can be indicated by prefixing"),a("br"),t._v("\nthe hash’s value with the algorithm name in lower-case. For example:")]),t._v(" "),a("div",{staticClass:"language- extra-class"},[a("pre",[a("code",[t._v('"hash": "sha1:8843d7f92416211de9ebb963ff4ce28125932878"\n')])])])]),t._v(" "),a("li",[a("p",[a("code",[t._v("sources")]),t._v(": as for "),a("RouterLink",{attrs:{to:"/data-package/"}},[t._v("Data Package metadata")]),t._v(".")],1)]),t._v(" "),a("li",[a("p",[a("code",[t._v("licenses")]),t._v(": as for "),a("RouterLink",{attrs:{to:"/data-package/"}},[t._v("Data Package metadata")]),t._v(". If not specified the resource"),a("br"),t._v("\ninherits from the data package.")],1)])]),t._v(" "),a("h3",{attrs:{id:"resource-schemas"}},[a("a",{staticClass:"header-anchor",attrs:{href:"#resource-schemas"}},[t._v("#")]),t._v(" Resource Schemas")]),t._v(" "),a("p",[t._v("A Data Resource MAY have a "),a("code",[t._v("schema")]),t._v(" property to describe the schema of the resource data.")]),t._v(" "),a("p",[t._v("The value for the "),a("code",[t._v("schema")]),t._v(" property on a "),a("code",[t._v("resource")]),t._v(" MUST be an "),a("code",[t._v("object")]),t._v(" representing the schema OR a "),a("code",[t._v("string")]),t._v(" that identifies the location of the schema.")]),t._v(" "),a("p",[t._v("If a "),a("code",[t._v("string")]),t._v(" it must be a "),a("a",{attrs:{href:"#url-or-path"}},[t._v("url-or-path as defined above")]),t._v(", that is a fully qualified http URL or a relative POSIX path. The file at the location specified by this url-or-path string "),a("code",[t._v("MUST")]),t._v(" be a JSON document containing the schema.")]),t._v(" "),a("p",[t._v("NOTE: the Data Package specification places no restrictions on the form of the schema Object. This flexibility enables specific communities to define schemas appropriate for the data they manage. As an example, the "),a("RouterLink",{attrs:{to:"/tabular-data-package/"}},[t._v("Tabular Data Package")]),t._v(" specification requires the schema to conform to "),a("RouterLink",{attrs:{to:"/table-schema/"}},[t._v("Table Schema")]),t._v(".")],1)],1)}),[],!1,null,null,null);e.default=r.exports}}]); \ No newline at end of file diff --git a/assets/js/23.3ac20e0a.js b/assets/js/23.8e9ba058.js similarity index 99% rename from assets/js/23.3ac20e0a.js rename to assets/js/23.8e9ba058.js index ed2bfa35..b85645d7 100644 --- a/assets/js/23.3ac20e0a.js +++ b/assets/js/23.8e9ba058.js @@ -1 +1 @@ -(window.webpackJsonp=window.webpackJsonp||[]).push([[23],{395:function(e,a,t){"use strict";t.r(a);var r=t(26),i=Object(r.a)({},(function(){var e=this,a=e.$createElement,t=e._self._c||a;return t("ContentSlotsDistributor",{attrs:{"slot-key":e.$parent.slotKey}},[t("h1",{attrs:{id:"implementation"}},[t("a",{staticClass:"header-anchor",attrs:{href:"#implementation"}},[e._v("#")]),e._v(" Implementation")]),e._v(" "),t("p",[e._v("This document is meant to serve as an introduction and an entry point into writing a library that implements a Frictionless Data specification. The focus is on two libraries in particular - Data Package and Table Schema - as implementing these libraries essentially implements the whole family of specifications.")]),e._v(" "),t("p",[e._v("The reader, being an implementer/maintainer of such libraries, should get a clear understanding of the reference material available for undertaking work, and the minimal set of "),t("strong",[e._v("actions")]),e._v(" that such libraries must enable for their users.")]),e._v(" "),t("p",[e._v("We prefer to focus on "),t("strong",[e._v("actions")]),e._v(" rather than features, feature sets, user stories, or more formal API specifications as we want to leave enough flexibility for implementations that follow the idioms of the host language, yet we do want to ensure a common base of "),t("em",[e._v("what can be done")]),e._v(" with an implementation in any language.")]),e._v(" "),t("h2",{attrs:{id:"context"}},[t("a",{staticClass:"header-anchor",attrs:{href:"#context"}},[e._v("#")]),e._v(" Context")]),e._v(" "),t("p",[e._v("While OKI and various other 3rd parties have been using the "),t("RouterLink",{attrs:{to:"/"}},[e._v("Data Package family of specifications")]),e._v(" with great success for several years, it has mostly been over the last 12 months that we are starting to see more mature libraries to implement the specifications at a “low level” for ease of reuse.")],1),e._v(" "),t("p",[e._v("At present, we consider the libraries maintained by Open Knowledge International in both Python and JavaScript to be reference implementations that serve as a guide for how to approach the specifications in code, and the type of actions that are enabled for users of the libraries. Further, we make extensive use of "),t("a",{attrs:{href:"http://json-schema.org",target:"_blank",rel:"noopener noreferrer"}},[e._v("JSON Schema"),t("OutboundLink")],1),e._v(" to validate "),t("code",[e._v("descriptors")]),e._v(" that are passed to these libraries. This enables significant reuse across implementations for descriptor validation logic.")]),e._v(" "),t("h2",{attrs:{id:"high-level-requirements"}},[t("a",{staticClass:"header-anchor",attrs:{href:"#high-level-requirements"}},[e._v("#")]),e._v(" High-level requirements")]),e._v(" "),t("p",[e._v("Here we will describe the minimal requirements for implementing Frictionless Data specifications in a given programming language. Based on the Python and JavaScript implementations, the implementations are split across two packages: "),t("code",[e._v("Data Package")]),e._v(" and "),t("code",[e._v("Table Schema")]),e._v(".")]),e._v(" "),t("p",[e._v("Also, see the "),t("a",{attrs:{href:"https://github.com/frictionlessdata/stack/blob/master/README.md",target:"_blank",rel:"noopener noreferrer"}},[e._v("stack reference"),t("OutboundLink")],1),e._v(" section below for some naming conventions we use, and that ideally should be followed in new implementations.")]),e._v(" "),t("h3",{attrs:{id:"data-package-library"}},[t("a",{staticClass:"header-anchor",attrs:{href:"#data-package-library"}},[e._v("#")]),e._v(" Data Package library")]),e._v(" "),t("p",[e._v("The Data Package library can load and validate any "),t("code",[e._v("descriptor")]),e._v(" for a "),t("strong",[e._v("Data Package Profile")]),e._v(", allow the creation and modification of "),t("code",[e._v("descriptors")]),e._v(", and expose methods for reading and streaming data in the package. When a "),t("code",[e._v("descriptor")]),e._v(" is a Tabular Data Package, it uses the Table Schema library, and exposes its functionality, for each "),t("code",[e._v("resource")]),e._v(" object in the "),t("code",[e._v("resources")]),e._v(" array.")]),e._v(" "),t("h4",{attrs:{id:"references"}},[t("a",{staticClass:"header-anchor",attrs:{href:"#references"}},[e._v("#")]),e._v(" References")]),e._v(" "),t("ul",[t("li",[t("RouterLink",{attrs:{to:"/data-package/"}},[e._v("Data Package specification")])],1),e._v(" "),t("li",[t("RouterLink",{attrs:{to:"/data-package/#profiles"}},[e._v("Data Package Profiles")])],1),e._v(" "),t("li",[t("RouterLink",{attrs:{to:"/tabular-data-package/"}},[e._v("Tabular Data Package specification (the most commonly used and useful Profile)")])],1),e._v(" "),t("li",[t("a",{attrs:{href:"/schemas/registry.json"}},[e._v("JSON Schema Registry")])]),e._v(" "),t("li",[t("a",{attrs:{href:"https://github.com/frictionlessdata/datapackage-js",target:"_blank",rel:"noopener noreferrer"}},[e._v("JavaScript implementation"),t("OutboundLink")],1)]),e._v(" "),t("li",[t("a",{attrs:{href:"https://github.com/frictionlessdata/datapackage-py",target:"_blank",rel:"noopener noreferrer"}},[e._v("Python implementation"),t("OutboundLink")],1)]),e._v(" "),t("li",[t("a",{attrs:{href:"https://github.com/christophergandrud/dpmr",target:"_blank",rel:"noopener noreferrer"}},[e._v("R implementation"),t("OutboundLink")],1)])]),e._v(" "),t("h4",{attrs:{id:"actions"}},[t("a",{staticClass:"header-anchor",attrs:{href:"#actions"}},[e._v("#")]),e._v(" Actions")]),e._v(" "),t("ul",[t("li",[e._v("read an existing Data Package descriptor")]),e._v(" "),t("li",[e._v("validate an existing Data Package descriptor, including profile-specific validation via the registry of JSON Schemas")]),e._v(" "),t("li",[e._v("create a new Data Package descriptor")]),e._v(" "),t("li",[e._v("edit an existing Data Package descriptor")]),e._v(" "),t("li",[e._v("as part of editing a descriptor, helper methods to add and remove resources from the resources array")]),e._v(" "),t("li",[e._v("validate edits made to a data package descriptor")]),e._v(" "),t("li",[e._v("save a Data Package descriptor to a file path")]),e._v(" "),t("li",[e._v("zip a Data Package descriptor and its co-located references (more generically: “zip a data package”)")]),e._v(" "),t("li",[e._v("read a zip file that “claims” to be a data package")]),e._v(" "),t("li",[e._v("save a zipped Data Package to disk")])]),e._v(" "),t("h4",{attrs:{id:"examples"}},[t("a",{staticClass:"header-anchor",attrs:{href:"#examples"}},[e._v("#")]),e._v(" Examples")]),e._v(" "),t("ul",[t("li",[t("a",{attrs:{href:"https://github.com/frictionlessdata/datapackage-js/blob/master/src/package.js",target:"_blank",rel:"noopener noreferrer"}},[e._v("Datapackage"),t("OutboundLink")],1)]),e._v(" "),t("li",[t("a",{attrs:{href:"https://github.com/frictionlessdata/datapackage-js/blob/master/src/resource.js",target:"_blank",rel:"noopener noreferrer"}},[e._v("Resource"),t("OutboundLink")],1)]),e._v(" "),t("li",[t("a",{attrs:{href:"https://github.com/frictionlessdata/datapackage-js/blob/master/src/validate.js",target:"_blank",rel:"noopener noreferrer"}},[e._v("validate"),t("OutboundLink")],1)])]),e._v(" "),t("h3",{attrs:{id:"table-schema-library"}},[t("a",{staticClass:"header-anchor",attrs:{href:"#table-schema-library"}},[e._v("#")]),e._v(" Table Schema library")]),e._v(" "),t("p",[e._v("The Table Schema library can load and validate any Table Schema "),t("code",[e._v("descriptor")]),e._v(", allow the creation and modification of "),t("code",[e._v("descriptors")]),e._v(", expose methods for reading and streaming data that conforms to a Table Schema via the Tabular Data Resource abstraction.")]),e._v(" "),t("h4",{attrs:{id:"references-2"}},[t("a",{staticClass:"header-anchor",attrs:{href:"#references-2"}},[e._v("#")]),e._v(" References")]),e._v(" "),t("ul",[t("li",[t("RouterLink",{attrs:{to:"/table-schema/"}},[e._v("Table Schema specification")])],1),e._v(" "),t("li",[t("RouterLink",{attrs:{to:"/tabular-data-resource/"}},[e._v("Tabular Data Resource specification")])],1),e._v(" "),t("li",[t("a",{attrs:{href:"/schemas/registry.json"}},[e._v("JSON Schema Registry")])]),e._v(" "),t("li",[t("a",{attrs:{href:"https://github.com/frictionlessdata/tableschema-js",target:"_blank",rel:"noopener noreferrer"}},[e._v("JavaScript implementation"),t("OutboundLink")],1)]),e._v(" "),t("li",[t("a",{attrs:{href:"https://github.com/frictionlessdata/tableschema-py",target:"_blank",rel:"noopener noreferrer"}},[e._v("Python implementation"),t("OutboundLink")],1)])]),e._v(" "),t("h4",{attrs:{id:"actions-2"}},[t("a",{staticClass:"header-anchor",attrs:{href:"#actions-2"}},[e._v("#")]),e._v(" Actions")]),e._v(" "),t("ul",[t("li",[e._v("read an existing Table Schema descriptor")]),e._v(" "),t("li",[e._v("validate an existing Table Schema descriptor using the JSON Schema spec")]),e._v(" "),t("li",[e._v("create a new Table Schema descriptor")]),e._v(" "),t("li",[e._v("edit an existing Table Schema descriptor")]),e._v(" "),t("li",[e._v("provide a model-type interface to interact with a descriptor")]),e._v(" "),t("li",[e._v("infer a Table Schema descriptor from a supplied sample of data")]),e._v(" "),t("li",[e._v("validate a data source against the Table Schema descriptor, including in response to editing the descriptor")]),e._v(" "),t("li",[e._v("enable streaming and reading of a data source "),t("em",[e._v("through")]),e._v(" a Table Schema (cast on iteration)")])]),e._v(" "),t("h4",{attrs:{id:"examples-2"}},[t("a",{staticClass:"header-anchor",attrs:{href:"#examples-2"}},[e._v("#")]),e._v(" Examples")]),e._v(" "),t("ul",[t("li",[t("a",{attrs:{href:"https://github.com/frictionlessdata/tableschema-py/blob/master/tableschema/table.py",target:"_blank",rel:"noopener noreferrer"}},[e._v("Table"),t("OutboundLink")],1)]),e._v(" "),t("li",[t("a",{attrs:{href:"https://github.com/frictionlessdata/tableschema-py/blob/master/tableschema/schema.py",target:"_blank",rel:"noopener noreferrer"}},[e._v("Schema"),t("OutboundLink")],1)]),e._v(" "),t("li",[t("a",{attrs:{href:"https://github.com/frictionlessdata/tableschema-py/blob/master/tableschema/field.py",target:"_blank",rel:"noopener noreferrer"}},[e._v("Field"),t("OutboundLink")],1)]),e._v(" "),t("li",[t("a",{attrs:{href:"https://github.com/frictionlessdata/tableschema-py/blob/master/tableschema/validate.py",target:"_blank",rel:"noopener noreferrer"}},[e._v("validate"),t("OutboundLink")],1)]),e._v(" "),t("li",[t("a",{attrs:{href:"https://github.com/frictionlessdata/tableschema-py/blob/master/tableschema/infer.py",target:"_blank",rel:"noopener noreferrer"}},[e._v("infer"),t("OutboundLink")],1)])]),e._v(" "),t("h3",{attrs:{id:"on-dereferencing-and-descriptor-validation"}},[t("a",{staticClass:"header-anchor",attrs:{href:"#on-dereferencing-and-descriptor-validation"}},[e._v("#")]),e._v(" On dereferencing and descriptor validation")]),e._v(" "),t("p",[e._v("Some properties in the Frictionless Data specifications allow a path (a URL or a POSIX path) that resolves to an object.")]),e._v(" "),t("p",[e._v("The most prominent example of this is the "),t("code",[e._v("schema")]),e._v(" property on Tabular Data Resource descriptors.")]),e._v(" "),t("p",[e._v("Allowing such references has practical use for publishers, for example in allowing schema reuse. However, it does introduce difficulties in the validation of such properties. For example, validating a path pointing to a schema rather than the schema object itself will do little to guarantee the integrity of the schema definition. Therefore implementors "),t("code",[e._v("MUST")]),e._v(" dereference such “referential” property values before attempting to validate a descriptor. At present, this requirement applies to the following properties in Tabular Data Package and Tabular Data Resource:")]),e._v(" "),t("ul",[t("li",[t("code",[e._v("schema")])]),e._v(" "),t("li",[t("code",[e._v("dialect")])])]),e._v(" "),t("h3",{attrs:{id:"other-libraries"}},[t("a",{staticClass:"header-anchor",attrs:{href:"#other-libraries"}},[e._v("#")]),e._v(" Other libraries")]),e._v(" "),t("p",[e._v("Data Package and Table Schema implement the core Frictionless Data specifications. The JavaScript implementations maintained by Open Knowledge International essentially follow the above requirements as is. However, our Python toolchain has some additional libraries - "),t("code",[e._v("goodtables")]),e._v(" and "),t("code",[e._v("tabulator")]),e._v(" - which are an important part of the Frictionless Data stack, and we would be delighted to see them implemented in other languages either as standalone libraries, or, as part of a wider effort in implementing Data Package and Table Schema.")]),e._v(" "),t("h4",{attrs:{id:"tabulator"}},[t("a",{staticClass:"header-anchor",attrs:{href:"#tabulator"}},[e._v("#")]),e._v(" tabulator")]),e._v(" "),t("p",[e._v("tabulator provides a consistent interface for streaming reading and writing of tabular data. It supports CSV, which is required for Table Schema, Tabular Data Resource, and Tabular Data Package, and also supports Excel, JSON, newline delimited JSON, Google Sheets, and ODS.")]),e._v(" "),t("h5",{attrs:{id:"references-3"}},[t("a",{staticClass:"header-anchor",attrs:{href:"#references-3"}},[e._v("#")]),e._v(" References")]),e._v(" "),t("ul",[t("li",[t("a",{attrs:{href:"https://github.com/frictionlessdata/tabulator-py",target:"_blank",rel:"noopener noreferrer"}},[e._v("Python implementation"),t("OutboundLink")],1)])]),e._v(" "),t("h4",{attrs:{id:"goodtables"}},[t("a",{staticClass:"header-anchor",attrs:{href:"#goodtables"}},[e._v("#")]),e._v(" goodtables")]),e._v(" "),t("p",[e._v("goodtables validates tabular data, checking for structural and schematic errors, and producing reports that can be used to iterate on data file sources as part of common data publication work flows. goodtables uses "),t("code",[e._v("tabulator")]),e._v(", "),t("code",[e._v("tableschema")]),e._v(", and "),t("code",[e._v("datapackage")]),e._v(" internally, as well as implementing "),t("code",[e._v("data-quality-spec")]),e._v(".")]),e._v(" "),t("p",[e._v("It may be of general interest that "),t("code",[e._v("goodtables")]),e._v(" is also available as a service - "),t("a",{attrs:{href:"https://goodtables.io",target:"_blank",rel:"noopener noreferrer"}},[t("code",[e._v("goodtables.io")]),t("OutboundLink")],1),e._v(" - providing continuous data validation in the style of CI solutions for code.")]),e._v(" "),t("h5",{attrs:{id:"references-4"}},[t("a",{staticClass:"header-anchor",attrs:{href:"#references-4"}},[e._v("#")]),e._v(" References")]),e._v(" "),t("ul",[t("li",[t("a",{attrs:{href:"https://github.com/frictionlessdata/goodtables-py",target:"_blank",rel:"noopener noreferrer"}},[e._v("Python implementation"),t("OutboundLink")],1)]),e._v(" "),t("li",[t("a",{attrs:{href:"https://github.com/frictionlessdata/data-quality-spec",target:"_blank",rel:"noopener noreferrer"}},[e._v("Data Quality Spec"),t("OutboundLink")],1)])]),e._v(" "),t("h2",{attrs:{id:"work-process"}},[t("a",{staticClass:"header-anchor",attrs:{href:"#work-process"}},[e._v("#")]),e._v(" Work process")]),e._v(" "),t("p",[e._v("Open Knowledge International "),t("a",{attrs:{href:"https://github.com/okfn/coding-standards",target:"_blank",rel:"noopener noreferrer"}},[e._v("coding standards can be found here"),t("OutboundLink")],1),e._v(". While some of the standards document refers to idioms in JavaScript and Python, much of it is about more general standards around using git, testing requirements and so on. These need to be adhered to.")]),e._v(" "),t("p",[e._v("We have some small example code repositories in Python and JavaScript that demonstrate these coding standards.")]),e._v(" "),t("ul",[t("li",[t("a",{attrs:{href:"https://github.com/okfn/oki-py",target:"_blank",rel:"noopener noreferrer"}},[e._v("Python Package Example"),t("OutboundLink")],1)]),e._v(" "),t("li",[t("a",{attrs:{href:"https://github.com/okfn/oki-js",target:"_blank",rel:"noopener noreferrer"}},[e._v("JavaScript Package Example"),t("OutboundLink")],1)])]),e._v(" "),t("p",[e._v("If you would "),t("em",[e._v("like")]),e._v(" to contribute sections based on idioms in your target language, that would be great: it will serve as a further reference to others, and also have the added benefit of enabling our team to learn from you.")])])}),[],!1,null,null,null);a.default=i.exports}}]); \ No newline at end of file +(window.webpackJsonp=window.webpackJsonp||[]).push([[23],{393:function(e,a,t){"use strict";t.r(a);var r=t(26),i=Object(r.a)({},(function(){var e=this,a=e.$createElement,t=e._self._c||a;return t("ContentSlotsDistributor",{attrs:{"slot-key":e.$parent.slotKey}},[t("h1",{attrs:{id:"implementation"}},[t("a",{staticClass:"header-anchor",attrs:{href:"#implementation"}},[e._v("#")]),e._v(" Implementation")]),e._v(" "),t("p",[e._v("This document is meant to serve as an introduction and an entry point into writing a library that implements a Frictionless Data specification. The focus is on two libraries in particular - Data Package and Table Schema - as implementing these libraries essentially implements the whole family of specifications.")]),e._v(" "),t("p",[e._v("The reader, being an implementer/maintainer of such libraries, should get a clear understanding of the reference material available for undertaking work, and the minimal set of "),t("strong",[e._v("actions")]),e._v(" that such libraries must enable for their users.")]),e._v(" "),t("p",[e._v("We prefer to focus on "),t("strong",[e._v("actions")]),e._v(" rather than features, feature sets, user stories, or more formal API specifications as we want to leave enough flexibility for implementations that follow the idioms of the host language, yet we do want to ensure a common base of "),t("em",[e._v("what can be done")]),e._v(" with an implementation in any language.")]),e._v(" "),t("h2",{attrs:{id:"context"}},[t("a",{staticClass:"header-anchor",attrs:{href:"#context"}},[e._v("#")]),e._v(" Context")]),e._v(" "),t("p",[e._v("While OKI and various other 3rd parties have been using the "),t("RouterLink",{attrs:{to:"/"}},[e._v("Data Package family of specifications")]),e._v(" with great success for several years, it has mostly been over the last 12 months that we are starting to see more mature libraries to implement the specifications at a “low level” for ease of reuse.")],1),e._v(" "),t("p",[e._v("At present, we consider the libraries maintained by Open Knowledge International in both Python and JavaScript to be reference implementations that serve as a guide for how to approach the specifications in code, and the type of actions that are enabled for users of the libraries. Further, we make extensive use of "),t("a",{attrs:{href:"http://json-schema.org",target:"_blank",rel:"noopener noreferrer"}},[e._v("JSON Schema"),t("OutboundLink")],1),e._v(" to validate "),t("code",[e._v("descriptors")]),e._v(" that are passed to these libraries. This enables significant reuse across implementations for descriptor validation logic.")]),e._v(" "),t("h2",{attrs:{id:"high-level-requirements"}},[t("a",{staticClass:"header-anchor",attrs:{href:"#high-level-requirements"}},[e._v("#")]),e._v(" High-level requirements")]),e._v(" "),t("p",[e._v("Here we will describe the minimal requirements for implementing Frictionless Data specifications in a given programming language. Based on the Python and JavaScript implementations, the implementations are split across two packages: "),t("code",[e._v("Data Package")]),e._v(" and "),t("code",[e._v("Table Schema")]),e._v(".")]),e._v(" "),t("p",[e._v("Also, see the "),t("a",{attrs:{href:"https://github.com/frictionlessdata/stack/blob/master/README.md",target:"_blank",rel:"noopener noreferrer"}},[e._v("stack reference"),t("OutboundLink")],1),e._v(" section below for some naming conventions we use, and that ideally should be followed in new implementations.")]),e._v(" "),t("h3",{attrs:{id:"data-package-library"}},[t("a",{staticClass:"header-anchor",attrs:{href:"#data-package-library"}},[e._v("#")]),e._v(" Data Package library")]),e._v(" "),t("p",[e._v("The Data Package library can load and validate any "),t("code",[e._v("descriptor")]),e._v(" for a "),t("strong",[e._v("Data Package Profile")]),e._v(", allow the creation and modification of "),t("code",[e._v("descriptors")]),e._v(", and expose methods for reading and streaming data in the package. When a "),t("code",[e._v("descriptor")]),e._v(" is a Tabular Data Package, it uses the Table Schema library, and exposes its functionality, for each "),t("code",[e._v("resource")]),e._v(" object in the "),t("code",[e._v("resources")]),e._v(" array.")]),e._v(" "),t("h4",{attrs:{id:"references"}},[t("a",{staticClass:"header-anchor",attrs:{href:"#references"}},[e._v("#")]),e._v(" References")]),e._v(" "),t("ul",[t("li",[t("RouterLink",{attrs:{to:"/data-package/"}},[e._v("Data Package specification")])],1),e._v(" "),t("li",[t("RouterLink",{attrs:{to:"/data-package/#profiles"}},[e._v("Data Package Profiles")])],1),e._v(" "),t("li",[t("RouterLink",{attrs:{to:"/tabular-data-package/"}},[e._v("Tabular Data Package specification (the most commonly used and useful Profile)")])],1),e._v(" "),t("li",[t("a",{attrs:{href:"/schemas/registry.json"}},[e._v("JSON Schema Registry")])]),e._v(" "),t("li",[t("a",{attrs:{href:"https://github.com/frictionlessdata/datapackage-js",target:"_blank",rel:"noopener noreferrer"}},[e._v("JavaScript implementation"),t("OutboundLink")],1)]),e._v(" "),t("li",[t("a",{attrs:{href:"https://github.com/frictionlessdata/datapackage-py",target:"_blank",rel:"noopener noreferrer"}},[e._v("Python implementation"),t("OutboundLink")],1)]),e._v(" "),t("li",[t("a",{attrs:{href:"https://github.com/christophergandrud/dpmr",target:"_blank",rel:"noopener noreferrer"}},[e._v("R implementation"),t("OutboundLink")],1)])]),e._v(" "),t("h4",{attrs:{id:"actions"}},[t("a",{staticClass:"header-anchor",attrs:{href:"#actions"}},[e._v("#")]),e._v(" Actions")]),e._v(" "),t("ul",[t("li",[e._v("read an existing Data Package descriptor")]),e._v(" "),t("li",[e._v("validate an existing Data Package descriptor, including profile-specific validation via the registry of JSON Schemas")]),e._v(" "),t("li",[e._v("create a new Data Package descriptor")]),e._v(" "),t("li",[e._v("edit an existing Data Package descriptor")]),e._v(" "),t("li",[e._v("as part of editing a descriptor, helper methods to add and remove resources from the resources array")]),e._v(" "),t("li",[e._v("validate edits made to a data package descriptor")]),e._v(" "),t("li",[e._v("save a Data Package descriptor to a file path")]),e._v(" "),t("li",[e._v("zip a Data Package descriptor and its co-located references (more generically: “zip a data package”)")]),e._v(" "),t("li",[e._v("read a zip file that “claims” to be a data package")]),e._v(" "),t("li",[e._v("save a zipped Data Package to disk")])]),e._v(" "),t("h4",{attrs:{id:"examples"}},[t("a",{staticClass:"header-anchor",attrs:{href:"#examples"}},[e._v("#")]),e._v(" Examples")]),e._v(" "),t("ul",[t("li",[t("a",{attrs:{href:"https://github.com/frictionlessdata/datapackage-js/blob/master/src/package.js",target:"_blank",rel:"noopener noreferrer"}},[e._v("Datapackage"),t("OutboundLink")],1)]),e._v(" "),t("li",[t("a",{attrs:{href:"https://github.com/frictionlessdata/datapackage-js/blob/master/src/resource.js",target:"_blank",rel:"noopener noreferrer"}},[e._v("Resource"),t("OutboundLink")],1)]),e._v(" "),t("li",[t("a",{attrs:{href:"https://github.com/frictionlessdata/datapackage-js/blob/master/src/validate.js",target:"_blank",rel:"noopener noreferrer"}},[e._v("validate"),t("OutboundLink")],1)])]),e._v(" "),t("h3",{attrs:{id:"table-schema-library"}},[t("a",{staticClass:"header-anchor",attrs:{href:"#table-schema-library"}},[e._v("#")]),e._v(" Table Schema library")]),e._v(" "),t("p",[e._v("The Table Schema library can load and validate any Table Schema "),t("code",[e._v("descriptor")]),e._v(", allow the creation and modification of "),t("code",[e._v("descriptors")]),e._v(", expose methods for reading and streaming data that conforms to a Table Schema via the Tabular Data Resource abstraction.")]),e._v(" "),t("h4",{attrs:{id:"references-2"}},[t("a",{staticClass:"header-anchor",attrs:{href:"#references-2"}},[e._v("#")]),e._v(" References")]),e._v(" "),t("ul",[t("li",[t("RouterLink",{attrs:{to:"/table-schema/"}},[e._v("Table Schema specification")])],1),e._v(" "),t("li",[t("RouterLink",{attrs:{to:"/tabular-data-resource/"}},[e._v("Tabular Data Resource specification")])],1),e._v(" "),t("li",[t("a",{attrs:{href:"/schemas/registry.json"}},[e._v("JSON Schema Registry")])]),e._v(" "),t("li",[t("a",{attrs:{href:"https://github.com/frictionlessdata/tableschema-js",target:"_blank",rel:"noopener noreferrer"}},[e._v("JavaScript implementation"),t("OutboundLink")],1)]),e._v(" "),t("li",[t("a",{attrs:{href:"https://github.com/frictionlessdata/tableschema-py",target:"_blank",rel:"noopener noreferrer"}},[e._v("Python implementation"),t("OutboundLink")],1)])]),e._v(" "),t("h4",{attrs:{id:"actions-2"}},[t("a",{staticClass:"header-anchor",attrs:{href:"#actions-2"}},[e._v("#")]),e._v(" Actions")]),e._v(" "),t("ul",[t("li",[e._v("read an existing Table Schema descriptor")]),e._v(" "),t("li",[e._v("validate an existing Table Schema descriptor using the JSON Schema spec")]),e._v(" "),t("li",[e._v("create a new Table Schema descriptor")]),e._v(" "),t("li",[e._v("edit an existing Table Schema descriptor")]),e._v(" "),t("li",[e._v("provide a model-type interface to interact with a descriptor")]),e._v(" "),t("li",[e._v("infer a Table Schema descriptor from a supplied sample of data")]),e._v(" "),t("li",[e._v("validate a data source against the Table Schema descriptor, including in response to editing the descriptor")]),e._v(" "),t("li",[e._v("enable streaming and reading of a data source "),t("em",[e._v("through")]),e._v(" a Table Schema (cast on iteration)")])]),e._v(" "),t("h4",{attrs:{id:"examples-2"}},[t("a",{staticClass:"header-anchor",attrs:{href:"#examples-2"}},[e._v("#")]),e._v(" Examples")]),e._v(" "),t("ul",[t("li",[t("a",{attrs:{href:"https://github.com/frictionlessdata/tableschema-py/blob/master/tableschema/table.py",target:"_blank",rel:"noopener noreferrer"}},[e._v("Table"),t("OutboundLink")],1)]),e._v(" "),t("li",[t("a",{attrs:{href:"https://github.com/frictionlessdata/tableschema-py/blob/master/tableschema/schema.py",target:"_blank",rel:"noopener noreferrer"}},[e._v("Schema"),t("OutboundLink")],1)]),e._v(" "),t("li",[t("a",{attrs:{href:"https://github.com/frictionlessdata/tableschema-py/blob/master/tableschema/field.py",target:"_blank",rel:"noopener noreferrer"}},[e._v("Field"),t("OutboundLink")],1)]),e._v(" "),t("li",[t("a",{attrs:{href:"https://github.com/frictionlessdata/tableschema-py/blob/master/tableschema/validate.py",target:"_blank",rel:"noopener noreferrer"}},[e._v("validate"),t("OutboundLink")],1)]),e._v(" "),t("li",[t("a",{attrs:{href:"https://github.com/frictionlessdata/tableschema-py/blob/master/tableschema/infer.py",target:"_blank",rel:"noopener noreferrer"}},[e._v("infer"),t("OutboundLink")],1)])]),e._v(" "),t("h3",{attrs:{id:"on-dereferencing-and-descriptor-validation"}},[t("a",{staticClass:"header-anchor",attrs:{href:"#on-dereferencing-and-descriptor-validation"}},[e._v("#")]),e._v(" On dereferencing and descriptor validation")]),e._v(" "),t("p",[e._v("Some properties in the Frictionless Data specifications allow a path (a URL or a POSIX path) that resolves to an object.")]),e._v(" "),t("p",[e._v("The most prominent example of this is the "),t("code",[e._v("schema")]),e._v(" property on Tabular Data Resource descriptors.")]),e._v(" "),t("p",[e._v("Allowing such references has practical use for publishers, for example in allowing schema reuse. However, it does introduce difficulties in the validation of such properties. For example, validating a path pointing to a schema rather than the schema object itself will do little to guarantee the integrity of the schema definition. Therefore implementors "),t("code",[e._v("MUST")]),e._v(" dereference such “referential” property values before attempting to validate a descriptor. At present, this requirement applies to the following properties in Tabular Data Package and Tabular Data Resource:")]),e._v(" "),t("ul",[t("li",[t("code",[e._v("schema")])]),e._v(" "),t("li",[t("code",[e._v("dialect")])])]),e._v(" "),t("h3",{attrs:{id:"other-libraries"}},[t("a",{staticClass:"header-anchor",attrs:{href:"#other-libraries"}},[e._v("#")]),e._v(" Other libraries")]),e._v(" "),t("p",[e._v("Data Package and Table Schema implement the core Frictionless Data specifications. The JavaScript implementations maintained by Open Knowledge International essentially follow the above requirements as is. However, our Python toolchain has some additional libraries - "),t("code",[e._v("goodtables")]),e._v(" and "),t("code",[e._v("tabulator")]),e._v(" - which are an important part of the Frictionless Data stack, and we would be delighted to see them implemented in other languages either as standalone libraries, or, as part of a wider effort in implementing Data Package and Table Schema.")]),e._v(" "),t("h4",{attrs:{id:"tabulator"}},[t("a",{staticClass:"header-anchor",attrs:{href:"#tabulator"}},[e._v("#")]),e._v(" tabulator")]),e._v(" "),t("p",[e._v("tabulator provides a consistent interface for streaming reading and writing of tabular data. It supports CSV, which is required for Table Schema, Tabular Data Resource, and Tabular Data Package, and also supports Excel, JSON, newline delimited JSON, Google Sheets, and ODS.")]),e._v(" "),t("h5",{attrs:{id:"references-3"}},[t("a",{staticClass:"header-anchor",attrs:{href:"#references-3"}},[e._v("#")]),e._v(" References")]),e._v(" "),t("ul",[t("li",[t("a",{attrs:{href:"https://github.com/frictionlessdata/tabulator-py",target:"_blank",rel:"noopener noreferrer"}},[e._v("Python implementation"),t("OutboundLink")],1)])]),e._v(" "),t("h4",{attrs:{id:"goodtables"}},[t("a",{staticClass:"header-anchor",attrs:{href:"#goodtables"}},[e._v("#")]),e._v(" goodtables")]),e._v(" "),t("p",[e._v("goodtables validates tabular data, checking for structural and schematic errors, and producing reports that can be used to iterate on data file sources as part of common data publication work flows. goodtables uses "),t("code",[e._v("tabulator")]),e._v(", "),t("code",[e._v("tableschema")]),e._v(", and "),t("code",[e._v("datapackage")]),e._v(" internally, as well as implementing "),t("code",[e._v("data-quality-spec")]),e._v(".")]),e._v(" "),t("p",[e._v("It may be of general interest that "),t("code",[e._v("goodtables")]),e._v(" is also available as a service - "),t("a",{attrs:{href:"https://goodtables.io",target:"_blank",rel:"noopener noreferrer"}},[t("code",[e._v("goodtables.io")]),t("OutboundLink")],1),e._v(" - providing continuous data validation in the style of CI solutions for code.")]),e._v(" "),t("h5",{attrs:{id:"references-4"}},[t("a",{staticClass:"header-anchor",attrs:{href:"#references-4"}},[e._v("#")]),e._v(" References")]),e._v(" "),t("ul",[t("li",[t("a",{attrs:{href:"https://github.com/frictionlessdata/goodtables-py",target:"_blank",rel:"noopener noreferrer"}},[e._v("Python implementation"),t("OutboundLink")],1)]),e._v(" "),t("li",[t("a",{attrs:{href:"https://github.com/frictionlessdata/data-quality-spec",target:"_blank",rel:"noopener noreferrer"}},[e._v("Data Quality Spec"),t("OutboundLink")],1)])]),e._v(" "),t("h2",{attrs:{id:"work-process"}},[t("a",{staticClass:"header-anchor",attrs:{href:"#work-process"}},[e._v("#")]),e._v(" Work process")]),e._v(" "),t("p",[e._v("Open Knowledge International "),t("a",{attrs:{href:"https://github.com/okfn/coding-standards",target:"_blank",rel:"noopener noreferrer"}},[e._v("coding standards can be found here"),t("OutboundLink")],1),e._v(". While some of the standards document refers to idioms in JavaScript and Python, much of it is about more general standards around using git, testing requirements and so on. These need to be adhered to.")]),e._v(" "),t("p",[e._v("We have some small example code repositories in Python and JavaScript that demonstrate these coding standards.")]),e._v(" "),t("ul",[t("li",[t("a",{attrs:{href:"https://github.com/okfn/oki-py",target:"_blank",rel:"noopener noreferrer"}},[e._v("Python Package Example"),t("OutboundLink")],1)]),e._v(" "),t("li",[t("a",{attrs:{href:"https://github.com/okfn/oki-js",target:"_blank",rel:"noopener noreferrer"}},[e._v("JavaScript Package Example"),t("OutboundLink")],1)])]),e._v(" "),t("p",[e._v("If you would "),t("em",[e._v("like")]),e._v(" to contribute sections based on idioms in your target language, that would be great: it will serve as a further reference to others, and also have the added benefit of enabling our team to learn from you.")])])}),[],!1,null,null,null);a.default=i.exports}}]); \ No newline at end of file diff --git a/assets/js/24.d5b8135f.js b/assets/js/24.5225f1ba.js similarity index 99% rename from assets/js/24.d5b8135f.js rename to assets/js/24.5225f1ba.js index f80321a0..08cd935f 100644 --- a/assets/js/24.d5b8135f.js +++ b/assets/js/24.5225f1ba.js @@ -1 +1 @@ -(window.webpackJsonp=window.webpackJsonp||[]).push([[24],{393:function(e,t,a){"use strict";a.r(t);var s=a(26),n=Object(s.a)({},(function(){var e=this,t=e.$createElement,a=e._self._c||t;return a("ContentSlotsDistributor",{attrs:{"slot-key":e.$parent.slotKey}},[a("h1",{attrs:{id:"patterns"}},[a("a",{staticClass:"header-anchor",attrs:{href:"#patterns"}},[e._v("#")]),e._v(" Patterns")]),e._v(" "),a("p",[e._v("This document describes various patterns for solving common problems, in ways that are not (yet) specified in any Frictionless Data specification. If we see increased adoption, or wide support, for any pattern, it is a prime candidate for formalising as part of a specification.")]),e._v(" "),a("h2",{attrs:{id:"private-properties"}},[a("a",{staticClass:"header-anchor",attrs:{href:"#private-properties"}},[e._v("#")]),e._v(" Private properties")]),e._v(" "),a("h3",{attrs:{id:"overview"}},[a("a",{staticClass:"header-anchor",attrs:{href:"#overview"}},[e._v("#")]),e._v(" Overview")]),e._v(" "),a("p",[e._v("Some software that implements the Frictionless Data specifications may need to store additional information on the various Frictionless Data descriptors.")]),e._v(" "),a("p",[e._v("For example, a data registry that provides metadata via "),a("code",[e._v("datapackage.json")]),e._v(" may wish to set an internal version or identifier that is system-specific, and should not be considered as part of the user-generated metadata.")]),e._v(" "),a("p",[e._v("Properties to store such information should be considered “private”, and by convention, the names should be prefixed by an underscore "),a("code",[e._v("_")]),e._v(".")]),e._v(" "),a("h3",{attrs:{id:"implementations"}},[a("a",{staticClass:"header-anchor",attrs:{href:"#implementations"}},[e._v("#")]),e._v(" Implementations")]),e._v(" "),a("p",[e._v("There are no known implementations at present.")]),e._v(" "),a("h3",{attrs:{id:"specification"}},[a("a",{staticClass:"header-anchor",attrs:{href:"#specification"}},[e._v("#")]),e._v(" Specification")]),e._v(" "),a("p",[e._v("On any Frictionless Data descriptor, data that is not generated by the author/contributors, but is generated by software/a system handling the data, "),a("code",[e._v("SHOULD")]),e._v(" be considered as “private”, and be prefixed by an underscore "),a("code",[e._v("_")]),e._v(".")]),e._v(" "),a("p",[e._v("To demonstrate, let’s take the example of a data registry that implements "),a("code",[e._v("datapackage.json")]),e._v(" for storing dataset metadata.")]),e._v(" "),a("p",[e._v("A user might upload a "),a("code",[e._v("datapackage.json")]),e._v(" as follows:")]),e._v(" "),a("div",{staticClass:"language- extra-class"},[a("pre",{pre:!0,attrs:{class:"language-text"}},[a("code",[e._v('{\n "name": "my-package",\n "resources": [\n {\n "name": "my-resource",\n "data": [ "my-resource.csv" ]\n }\n ]\n}\n')])])]),a("p",[e._v("The registry itself may have a platform-specific version system, and increment versions on each update of the data. To store this information on the datapackage itself, the platform could save this information in a “private” "),a("code",[e._v("_platformVersion")]),e._v(" property as follows:")]),e._v(" "),a("div",{staticClass:"language- extra-class"},[a("pre",{pre:!0,attrs:{class:"language-text"}},[a("code",[e._v('{\n "name": "my-package",\n "_platformVersion": 7\n "resources": [\n {\n "name": "my-resource",\n "data": [ "my-resource.csv" ]\n }\n ]\n}\n')])])]),a("p",[e._v("Usage of “private” properties ensures a clear distinction between data stored on the descriptor that is user (author/contributor) defined, and any additional data that may be stored by a 3rd party.")]),e._v(" "),a("h2",{attrs:{id:"caching-of-resources"}},[a("a",{staticClass:"header-anchor",attrs:{href:"#caching-of-resources"}},[e._v("#")]),e._v(" Caching of resources")]),e._v(" "),a("h3",{attrs:{id:"overview-2"}},[a("a",{staticClass:"header-anchor",attrs:{href:"#overview-2"}},[e._v("#")]),e._v(" Overview")]),e._v(" "),a("p",[e._v("All Frictionless Data specifications allow for referencing resources via http or a local filesystem.")]),e._v(" "),a("p",[e._v("In the case of remote resources via http, there is always the possibility that the remote server will be unavailable, or, that the resource itself will be temporarily or permanently removed.")]),e._v(" "),a("p",[e._v("Applications that are concerned with the persistent storage of data described in Frictionless Data specifications can use a "),a("code",[e._v("_cache")]),e._v(" property that mirrors the functionality and usage of the "),a("code",[e._v("data")]),e._v(" property, and refers to a storage location for the data that the application can fall back to if the canonical resource is unavailable.")]),e._v(" "),a("h3",{attrs:{id:"implementations-2"}},[a("a",{staticClass:"header-anchor",attrs:{href:"#implementations-2"}},[e._v("#")]),e._v(" Implementations")]),e._v(" "),a("p",[e._v("There are no known implementations of this pattern at present.")]),e._v(" "),a("h3",{attrs:{id:"specification-2"}},[a("a",{staticClass:"header-anchor",attrs:{href:"#specification-2"}},[e._v("#")]),e._v(" Specification")]),e._v(" "),a("p",[e._v("Implementations "),a("code",[e._v("MAY")]),e._v(" handle a "),a("code",[e._v("_cache")]),e._v(" property on any descriptor that supports a "),a("code",[e._v("data")]),e._v(" property. In the case that the data referenced in "),a("code",[e._v("data")]),e._v(" is unavailable, "),a("code",[e._v("_cache")]),e._v(" should be used as a fallback to access the data. The handling of the data stored at "),a("code",[e._v("_cache")]),e._v(" is beyond the scope of the specification. Implementations might store a copy of the resources in "),a("code",[e._v("data")]),e._v(" at ingestion time, update at regular intervals, or any other method to keep an up-to-date, persistent copy.")]),e._v(" "),a("p",[e._v("Some examples of the "),a("code",[e._v("_cache")]),e._v(" property.")]),e._v(" "),a("div",{staticClass:"language- extra-class"},[a("pre",{pre:!0,attrs:{class:"language-text"}},[a("code",[e._v('{\n "name": "my-package",\n "resources": [\n {\n "name": "my-resource",\n "data": [ "http://example.com/data/csv/my-resource.csv" ],\n "_cache": "my-resource.csv"\n },\n {\n "name": "my-resource",\n "data": [ "http://example.com/data/csv/my-resource.csv" ],\n "_cache": "http://data.registry.com/user/files/my-resource.csv"\n },\n {\n "name": "my-resource",\n "data": [\n "http://example.com/data/csv/my-resource.csv",\n "http://somewhere-else.com/data/csv/resource2.csv"\n ],\n "_cache": [\n "my-resource.csv",\n "resource2.csv"\n ]\n },\n {\n "name": "my-resource",\n "data": [ "http://example.com/data/csv/my-resource.csv" ],\n "_cache": "my-resource.csv"\n }\n ]\n}\n')])])]),a("h2",{attrs:{id:"compression-of-resources"}},[a("a",{staticClass:"header-anchor",attrs:{href:"#compression-of-resources"}},[e._v("#")]),e._v(" Compression of resources")]),e._v(" "),a("h3",{attrs:{id:"overview-3"}},[a("a",{staticClass:"header-anchor",attrs:{href:"#overview-3"}},[e._v("#")]),e._v(" Overview")]),e._v(" "),a("p",[e._v("It can be argued that applying compression to data resources can make data package publishing more cost-effective and sustainable. Compressing data resources gives publishers the benefit of reduced storage and bandwidth costs and gives consumers the benefit of shorter download times.")]),e._v(" "),a("h3",{attrs:{id:"implementations-3"}},[a("a",{staticClass:"header-anchor",attrs:{href:"#implementations-3"}},[e._v("#")]),e._v(" Implementations")]),e._v(" "),a("ul",[a("li",[a("a",{attrs:{href:"https://github.com/frictionlessdata/tabulator-py",target:"_blank",rel:"noopener noreferrer"}},[e._v("tabulator-py (Gzip and Zip support)"),a("OutboundLink")],1)]),e._v(" "),a("li",[a("a",{attrs:{href:"https://github.com/nimblelearn/datapackage-connector",target:"_blank",rel:"noopener noreferrer"}},[e._v("datapackage-connector (Gzip support)"),a("OutboundLink")],1)]),e._v(" "),a("li",[a("a",{attrs:{href:"https://github.com/nimblelearn/datapackage-m",target:"_blank",rel:"noopener noreferrer"}},[e._v("datapackage-m (Gzip support)"),a("OutboundLink")],1)])]),e._v(" "),a("h3",{attrs:{id:"specification-3"}},[a("a",{staticClass:"header-anchor",attrs:{href:"#specification-3"}},[e._v("#")]),e._v(" Specification")]),e._v(" "),a("p",[e._v("All compressed resources "),a("code",[e._v("MUST")]),e._v(" have a "),a("code",[e._v("path")]),e._v(" that allows the "),a("code",[e._v("compression")]),e._v(" property to be inferred. If the compression can’t be inferred from the "),a("code",[e._v("path")]),e._v(" property (e.g. a custom file extension is used) then the "),a("code",[e._v("compression")]),e._v(" property "),a("code",[e._v("MUST")]),e._v(" be used to specify the compression.")]),e._v(" "),a("p",[e._v("Supported compression types:")]),e._v(" "),a("ul",[a("li",[e._v("gz")]),e._v(" "),a("li",[e._v("zip")])]),e._v(" "),a("p",[e._v("Example of a compressed resource with implied compression:")]),e._v(" "),a("div",{staticClass:"language- extra-class"},[a("pre",{pre:!0,attrs:{class:"language-text"}},[a("code",[e._v('{\n "name": "data-resource-compression-example",\n "path": "http://example.com/large-data-file.csv.gz",\n "title": "Large Data File",\n "description": "This large data file benefits from compression.",\n "format": "csv",\n "mediatype": "text/csv",\n "encoding": "utf-8",\n "bytes": 1073741824\n}\n')])])]),a("p",[e._v("Example of a compressed resource with the "),a("code",[e._v("compression")]),e._v(" property:")]),e._v(" "),a("div",{staticClass:"language- extra-class"},[a("pre",{pre:!0,attrs:{class:"language-text"}},[a("code",[e._v('{\n "name": "data-resource-compression-example",\n "path": "http://example.com/large-data-file.csv.gz",\n "title": "Large Data File",\n "description": "This large data file benefits from compression.",\n "format": "csv",\n "compression" : "gz",\n "mediatype": "text/csv",\n "encoding": "utf-8",\n "bytes": 1073741824\n}\n')])])]),a("div",{staticClass:"custom-block tip"},[a("p",{staticClass:"custom-block-title"},[e._v("NOTE")]),e._v(" "),a("p",[e._v("Resource properties e.g. bytes, hash etc apply to the compressed object – not to the original uncompressed object.")])]),e._v(" "),a("h2",{attrs:{id:"language-support"}},[a("a",{staticClass:"header-anchor",attrs:{href:"#language-support"}},[e._v("#")]),e._v(" Language support")]),e._v(" "),a("h3",{attrs:{id:"overview-4"}},[a("a",{staticClass:"header-anchor",attrs:{href:"#overview-4"}},[e._v("#")]),e._v(" Overview")]),e._v(" "),a("p",[e._v("Language support is a different concern to translation support. Language support deals with declaring the default language of a descriptor and the data it contains in the resources array. Language support makes no claim about the presence of translations when one or more languages are supported in a descriptor or in data. Via the introduction of a "),a("code",[e._v("languages")]),e._v(" array to any descriptor, we can declare the default language, and any other languages that "),a("code",[e._v("SHOULD")]),e._v(" be found in the descriptor and the data.")]),e._v(" "),a("h3",{attrs:{id:"implementations-4"}},[a("a",{staticClass:"header-anchor",attrs:{href:"#implementations-4"}},[e._v("#")]),e._v(" Implementations")]),e._v(" "),a("p",[e._v("There are no known implementations of this pattern at present.")]),e._v(" "),a("h3",{attrs:{id:"specification-4"}},[a("a",{staticClass:"header-anchor",attrs:{href:"#specification-4"}},[e._v("#")]),e._v(" Specification")]),e._v(" "),a("p",[e._v("Any Frictionless Data descriptor can declare the language configuration of its metadata and data with the "),a("code",[e._v("languages")]),e._v(" array.")]),e._v(" "),a("p",[a("code",[e._v("languages")]),e._v(" "),a("code",[e._v("MUST")]),e._v(" be an array, and the first item in the array is the default (non-translated) language.")]),e._v(" "),a("p",[e._v("If no "),a("code",[e._v("languages")]),e._v(" array is present, the default language is English ("),a("code",[e._v("en")]),e._v("), and therefore is equivalent to:")]),e._v(" "),a("div",{staticClass:"language- extra-class"},[a("pre",{pre:!0,attrs:{class:"language-text"}},[a("code",[e._v('{\n "name": "my-package",\n "languages": ["en"]\n}\n')])])]),a("p",[e._v("The presence of a languages array does not ensure that the metadata or the data has translations for all supported languages.")]),e._v(" "),a("p",[e._v("The descriptor and data sources "),a("code",[e._v("MUST")]),e._v(" be in the default language. The descriptor and data sources "),a("code",[e._v("MAY")]),e._v(" have translations for the other languages in the array, using the same language code. "),a("code",[e._v("IF")]),e._v(" a translation is not present, implementing code "),a("code",[e._v("MUST")]),e._v(" fallback to the default language string.")]),e._v(" "),a("p",[e._v("Example usage of "),a("code",[e._v("languages")]),e._v(", implemented in the metadata of a descriptor:")]),e._v(" "),a("div",{staticClass:"language- extra-class"},[a("pre",{pre:!0,attrs:{class:"language-text"}},[a("code",[e._v('{\n "name": "sun-package",\n "languages": ["es", "en"],\n "title": "Sol"\n}\n\n# which is equivalent to\n{\n "name": "sun-package",\n "languages": ["es", "en"],\n "title": {\n "": "Sol",\n "en": "Sun"\n }\n}\n')])])]),a("p",[e._v("Example usage of "),a("code",[e._v("languages")]),e._v(" implemented in the data described by a resource:")]),e._v(" "),a("div",{staticClass:"language- extra-class"},[a("pre",{pre:!0,attrs:{class:"language-text"}},[a("code",[e._v('# resource descriptor\n{\n "name": "solar-system",\n "data": [ "solar-system.csv" ]\n "fields": [\n ...\n ],\n "languages": ["es", "en", "he", "fr", "ar"]\n}\n\n# data source\n# some languages have translations, some do not\n# assumes a certain translation pattern, see the related section\nid,name,name@fr,name@he,name@en\n1,Sol,Soleil,שמש,Sun\n2,Luna,Lune,ירח,Moon\n')])])]),a("h2",{attrs:{id:"translation-support"}},[a("a",{staticClass:"header-anchor",attrs:{href:"#translation-support"}},[e._v("#")]),e._v(" Translation support")]),e._v(" "),a("h3",{attrs:{id:"overview-5"}},[a("a",{staticClass:"header-anchor",attrs:{href:"#overview-5"}},[e._v("#")]),e._v(" Overview")]),e._v(" "),a("p",[e._v("Following on from a general pattern for language support, and the explicit support of metadata translations in Frictionless Data descriptors, it would be desirable to support translations in source data.")]),e._v(" "),a("p",[e._v("We currently have two patterns for this in discussion. Both patterns arise from real-world implementations that are not specifically tied to Frictionless Data.")]),e._v(" "),a("p",[e._v("One pattern suggests inline translations with the source data, reserving the "),a("code",[e._v("@")]),e._v(" symbol in the naming of fields to denote translations.")]),e._v(" "),a("p",[e._v("The other describes a pattern for storing additional translation sources, co-located with the “source” file described in a descriptor "),a("code",[e._v("data")]),e._v(".")]),e._v(" "),a("h3",{attrs:{id:"implementations-5"}},[a("a",{staticClass:"header-anchor",attrs:{href:"#implementations-5"}},[e._v("#")]),e._v(" Implementations")]),e._v(" "),a("p",[e._v("There are no known implementations of this pattern in the Frictionless Data core libraries at present.")]),e._v(" "),a("h3",{attrs:{id:"specification-5"}},[a("a",{staticClass:"header-anchor",attrs:{href:"#specification-5"}},[e._v("#")]),e._v(" Specification")]),e._v(" "),a("h4",{attrs:{id:"inline"}},[a("a",{staticClass:"header-anchor",attrs:{href:"#inline"}},[e._v("#")]),e._v(" Inline")]),e._v(" "),a("p",[a("strong",[e._v("Uses a column naming convention for accessing translations")]),e._v(".")]),e._v(" "),a("p",[e._v("Tabular resource descriptors support translations using "),a("code",[e._v("{field_name}@{lang_code}")]),e._v(" syntax for translated field names. "),a("code",[e._v("lang_code")]),e._v(" "),a("code",[e._v("MUST")]),e._v(" be present in the "),a("code",[e._v("languages")]),e._v(" array that applies to the resource.")]),e._v(" "),a("p",[e._v("Any field with the "),a("code",[e._v("@")]),e._v(" symbol "),a("code",[e._v("MUST")]),e._v(" be a translation field for another field of data, and "),a("code",[e._v("MUST")]),e._v(" be parsable according to the "),a("code",[e._v("{field_name}@{lang_code}")]),e._v(" pattern.")]),e._v(" "),a("p",[e._v("If a translation field is found in the data that does not have a corresponding "),a("code",[e._v("field")]),e._v(" (e.g.: "),a("code",[e._v("title@es")]),e._v(" but no "),a("code",[e._v("title")]),e._v("), then the translation field "),a("code",[e._v("SHOULD")]),e._v(" be ignored.")]),e._v(" "),a("p",[e._v("If a translation field is found in the data that uses a "),a("code",[e._v("lang_code")]),e._v(" "),a("em",[e._v("not")]),e._v(" declared in the applied "),a("code",[e._v("languages")]),e._v(" array, then the translation field "),a("code",[e._v("SHOULD")]),e._v(" be ignored.")]),e._v(" "),a("p",[e._v("Translation fields "),a("code",[e._v("MUST NOT")]),e._v(" be described in a schema "),a("code",[e._v("fields")]),e._v(" array.")]),e._v(" "),a("p",[e._v("Translation fields "),a("code",[e._v("MUST")]),e._v(" match the "),a("code",[e._v("type")]),e._v(", "),a("code",[e._v("format")]),e._v(" and "),a("code",[e._v("constraints")]),e._v(" of the field they translate, with a single exception: Translation fields are never required, and therefore "),a("code",[e._v("constraints.required")]),e._v(" is always "),a("code",[e._v("false")]),e._v(" for a translation field.")]),e._v(" "),a("h4",{attrs:{id:"co-located-translation-sources"}},[a("a",{staticClass:"header-anchor",attrs:{href:"#co-located-translation-sources"}},[e._v("#")]),e._v(" Co-located translation sources")]),e._v(" "),a("p",[a("strong",[e._v("Uses a file storage convention for accessing translations")]),e._v(".")]),e._v(" "),a("p",[e._v("To be contributed by @jheeffer")]),e._v(" "),a("ul",[a("li",[e._v("Has to handle local and remote resources")]),e._v(" "),a("li",[e._v("Has to be explicit about the translation key/value pattern in the translation files")])]),e._v(" "),a("div",{staticClass:"language- extra-class"},[a("pre",{pre:!0,attrs:{class:"language-text"}},[a("code",[e._v("# local\ndata/file1.csv\ndata/lang/file1-en.csv\ndata/lang/file1-es.csv\n\n# remote\nhttp://example/com/data/file2.csv\nhttp://example/com/data/lang/file2-en.csv\nhttp://example/com/data/lang/file2-es.csv\n")])])]),a("h2",{attrs:{id:"table-schema-foreign-keys-to-data-packages"}},[a("a",{staticClass:"header-anchor",attrs:{href:"#table-schema-foreign-keys-to-data-packages"}},[e._v("#")]),e._v(" Table Schema: Foreign Keys to Data Packages")]),e._v(" "),a("h3",{attrs:{id:"overview-6"}},[a("a",{staticClass:"header-anchor",attrs:{href:"#overview-6"}},[e._v("#")]),e._v(" Overview")]),e._v(" "),a("p",[e._v("A foreign key is a reference where values in a field (or fields) in a Tabular Data Resource link to values in a field (or fields) in a Tabular Data Resource in the same or in another Tabular Data Package.")]),e._v(" "),a("p",[e._v("This pattern allows users to link values in a field (or fields) in a Tabular Data Resource to values in a field (or fields) in a Tabular Data Resource in a different Tabular Data Package.")]),e._v(" "),a("h3",{attrs:{id:"specification-6"}},[a("a",{staticClass:"header-anchor",attrs:{href:"#specification-6"}},[e._v("#")]),e._v(" Specification")]),e._v(" "),a("p",[e._v("The "),a("RouterLink",{attrs:{to:"/table-schema/#foreign-keys"}},[a("code",[e._v("foreignKeys")])]),e._v(" array MAY have a property "),a("code",[e._v("package")]),e._v(". This property MUST be, either:")],1),e._v(" "),a("ul",[a("li",[e._v("a string that is a fully qualified HTTP address to a Data Package "),a("code",[e._v("datapackage.json")]),e._v(" file")]),e._v(" "),a("li",[e._v("a data package "),a("RouterLink",{attrs:{to:"/data-package/#name"}},[a("code",[e._v("name")])]),e._v(" that can be resolved by a canonical data package registry")],1)]),e._v(" "),a("p",[e._v("If the referenced data package has an "),a("RouterLink",{attrs:{to:"/data-package/#id"}},[a("code",[e._v("id")])]),e._v(" that is a fully qualified HTTP address, it SHOULD be used as the "),a("code",[e._v("package")]),e._v(" value.")],1),e._v(" "),a("p",[e._v("For example:")]),e._v(" "),a("div",{staticClass:"language- extra-class"},[a("pre",{pre:!0,attrs:{class:"language-text"}},[a("code",[e._v('"foreignKeys": [{\n "fields": ["code"],\n "reference": {\n "package": "https://raw.githubusercontent.com/frictionlessdata/example-data-packages/master/donation-codes/datapackage.json",\n "resource": "donation-codes",\n "fields": ["donation code"]\n }\n }]\n')])])]),a("h2",{attrs:{id:"data-package-version"}},[a("a",{staticClass:"header-anchor",attrs:{href:"#data-package-version"}},[e._v("#")]),e._v(" Data Package Version")]),e._v(" "),a("h3",{attrs:{id:"specification-7"}},[a("a",{staticClass:"header-anchor",attrs:{href:"#specification-7"}},[e._v("#")]),e._v(" Specification")]),e._v(" "),a("p",[e._v("The Data Package version format follows the "),a("a",{attrs:{href:"http://semver.org",target:"_blank",rel:"noopener noreferrer"}},[e._v("Semantic Versioning"),a("OutboundLink")],1),e._v(" specification format: MAJOR.MINOR.PATCH")]),e._v(" "),a("p",[e._v("The version numbers, and the way they change, convey meaning about how the data package has been modified from one version to the next.")]),e._v(" "),a("p",[e._v("Given a Data Package version number MAJOR.MINOR.PATCH, increment the:")]),e._v(" "),a("p",[e._v("MAJOR version when you make incompatible changes, e.g.")]),e._v(" "),a("ul",[a("li",[e._v("Change the data package, resource or field "),a("code",[e._v("name")]),e._v(" or "),a("code",[e._v("identifier")])]),e._v(" "),a("li",[e._v("Add, remove or re-order fields")]),e._v(" "),a("li",[e._v("Change a field "),a("code",[e._v("type")]),e._v(" or "),a("code",[e._v("format")])]),e._v(" "),a("li",[e._v("Change a field "),a("code",[e._v("constraint")]),e._v(" to be more restrictive")]),e._v(" "),a("li",[e._v("Combine, split, delete or change the meaning of data that is referenced by another data resource")])]),e._v(" "),a("p",[e._v("MINOR version when you add data or change metadata in a backwards-compatible manner, e.g.")]),e._v(" "),a("ul",[a("li",[e._v("Add a new data resource to a data package")]),e._v(" "),a("li",[e._v("Add new data to an existing data resource")]),e._v(" "),a("li",[e._v("Change a field "),a("code",[e._v("constraint")]),e._v(" to be less restrictive")]),e._v(" "),a("li",[e._v("Update a reference to another data resource")]),e._v(" "),a("li",[e._v("Change data to reflect changes in referenced data")])]),e._v(" "),a("p",[e._v("PATCH version when you make backwards-compatible fixes, e.g.")]),e._v(" "),a("ul",[a("li",[e._v("Correct errors in existing data")]),e._v(" "),a("li",[e._v("Change descriptive metadata properties")])]),e._v(" "),a("h3",{attrs:{id:"scenarios"}},[a("a",{staticClass:"header-anchor",attrs:{href:"#scenarios"}},[e._v("#")]),e._v(" Scenarios")]),e._v(" "),a("ul",[a("li",[e._v("You are developing your data though public consultation. Start your initial data release at 0.1.0")]),e._v(" "),a("li",[e._v("You release your data for the first time. Use version 1.0.0")]),e._v(" "),a("li",[e._v("You append last months data to an existing release. Increment the MINOR version number")]),e._v(" "),a("li",[e._v("You append a column to the data. Increment the MAJOR version number")]),e._v(" "),a("li",[e._v("You relocate the data to a new "),a("code",[e._v("URL")]),e._v(" or "),a("code",[e._v("path")]),e._v(". No change in the version number")]),e._v(" "),a("li",[e._v("You change a "),a("code",[e._v("title")]),e._v(", "),a("code",[e._v("description")]),e._v(", or other descriptive metadata. Increment the PATCH version")]),e._v(" "),a("li",[e._v("You fix a data entry error by modifying a value. Increment the PATCH version")]),e._v(" "),a("li",[e._v("You split a row of data in a foreign key reference table. Increment the MAJOR version number")]),e._v(" "),a("li",[e._v("You update the data and schema to refer to a new version of a foreign key reference table. Increment the MINOR version number")])]),e._v(" "),a("h2",{attrs:{id:"data-dependencies"}},[a("a",{staticClass:"header-anchor",attrs:{href:"#data-dependencies"}},[e._v("#")]),e._v(" Data Dependencies")]),e._v(" "),a("p",[e._v("Consider a situation where data packages are part of a tool chain that, say, loads all of the data into an SQL db. You can then imagine a situation where one requires package A which requires package B + C.")]),e._v(" "),a("p",[e._v("In this case you want to specify that A depends on B and C – and that “installing” A should install B and C. This is the purpose of "),a("code",[e._v("dataDependencies")]),e._v(" property.")]),e._v(" "),a("h3",{attrs:{id:"specification-8"}},[a("a",{staticClass:"header-anchor",attrs:{href:"#specification-8"}},[e._v("#")]),e._v(" Specification")]),e._v(" "),a("p",[a("code",[e._v("dataDependencies")]),e._v(" is an object. It follows same format as CommonJS Packages spec v1.1. Each dependency defines the lowest compatible MAJOR[.MINOR[.PATCH]] dependency versions (only one per MAJOR version) with which the package has been tested and is assured to work. The version may be a simple version string (see the version property for acceptable forms), or it may be an object group of dependencies which define a set of options, any one of which satisfies the dependency. The ordering of the group is significant and earlier entries have higher priority. Example:")]),e._v(" "),a("div",{staticClass:"language-javascript extra-class"},[a("pre",{pre:!0,attrs:{class:"language-javascript"}},[a("code",[a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"dataDependencies"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("{")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"country-codes"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('""')]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v(",")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"unemployment"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"2.1"')]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v(",")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"geo-boundaries"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("{")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"acmecorp-geo-boundaries"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("[")]),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"1.0"')]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v(",")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"2.0"')]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("]")]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v(",")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"othercorp-geo-boundaries"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"0.9.8"')]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v(",")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("}")]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v(",")]),e._v("\n"),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("}")]),e._v("\n")])])]),a("h3",{attrs:{id:"implementations-6"}},[a("a",{staticClass:"header-anchor",attrs:{href:"#implementations-6"}},[e._v("#")]),e._v(" Implementations")]),e._v(" "),a("p",[e._v("None known.")]),e._v(" "),a("h2",{attrs:{id:"table-schema-metadata-properties"}},[a("a",{staticClass:"header-anchor",attrs:{href:"#table-schema-metadata-properties"}},[e._v("#")]),e._v(" Table Schema: metadata properties")]),e._v(" "),a("h3",{attrs:{id:"overview-7"}},[a("a",{staticClass:"header-anchor",attrs:{href:"#overview-7"}},[e._v("#")]),e._v(" Overview")]),e._v(" "),a("p",[e._v("Table Schemas need their own metadata to be stand-alone and interpreted without relying on other contextual information (Data Package metadata for example). Adding metadata to describe schemas in a structured way would help users to understand them and would increase their sharing and reuse.")]),e._v(" "),a("p",[e._v("Currently it is possible to add custom properties to a Table Schema, but the lack of consensus about those properties restricts common tooling and wider adoption.")]),e._v(" "),a("h3",{attrs:{id:"use-cases"}},[a("a",{staticClass:"header-anchor",attrs:{href:"#use-cases"}},[e._v("#")]),e._v(" Use cases")]),e._v(" "),a("ul",[a("li",[e._v("Documentation: generating Markdown documentation from the schema itself is a useful use case, and contextual information (description, version, authors…) needs to be retrieved.")]),e._v(" "),a("li",[e._v("Cataloging: open data standardisation can be increased by improving Table Schemas shareability, for example by searching and categorising them (by keywords, countries, full-text…) in catalogs.")]),e._v(" "),a("li",[e._v("Machine readability: tools like Goodtables could use catalogs to access Table Schemas in order to help users validate tabular files against existing schemas. Metadata would be needed for tools to find and read those schemas.")])]),e._v(" "),a("h3",{attrs:{id:"specification-9"}},[a("a",{staticClass:"header-anchor",attrs:{href:"#specification-9"}},[e._v("#")]),e._v(" Specification")]),e._v(" "),a("p",[e._v("This pattern introduces the following properties to the Table Schema spec (using "),a("a",{attrs:{href:"https://github.com/frictionlessdata/specs/blob/master/schemas/dictionary/common.yml",target:"_blank",rel:"noopener noreferrer"}},[e._v("the Frictionless Data core dictionary"),a("OutboundLink")],1),e._v(" as much as possible):")]),e._v(" "),a("ul",[a("li",[a("code",[e._v("name")]),e._v(": An identifier string for this schema.")]),e._v(" "),a("li",[a("code",[e._v("title")]),e._v(": A human-readable title for this schema.")]),e._v(" "),a("li",[a("code",[e._v("description")]),e._v(": A text description for this schema.")]),e._v(" "),a("li",[a("code",[e._v("keywords")]),e._v(": The keyword(s) that describe this schema."),a("br"),e._v(" "),a("em",[e._v("Tags are useful to categorise and catalog schemas.")])]),e._v(" "),a("li",[a("code",[e._v("countryCode")]),e._v(": The ISO 3166-1 alpha-2 code for the country where this schema is primarily used."),a("br"),e._v(" "),a("em",[e._v("Since open data schemas are very country-specific, it’s useful to have this information in a structured way.")])]),e._v(" "),a("li",[a("code",[e._v("homepage")]),e._v(": The home on the web that is related to this schema.")]),e._v(" "),a("li",[a("code",[e._v("path")]),e._v(": A fully qualified URL for this schema."),a("br"),e._v(" "),a("em",[e._v("The direct path to the schema itself can be useful to help accessing it (i.e. machine readability).")])]),e._v(" "),a("li",[a("code",[e._v("image")]),e._v(": An image to represent this schema."),a("br"),e._v(" "),a("em",[e._v("An optional illustration can be useful for example in catalogs to differentiate schemas in a list.")])]),e._v(" "),a("li",[a("code",[e._v("licenses")]),e._v(": The license(s) under which this schema is published.")]),e._v(" "),a("li",[a("code",[e._v("resources")]),e._v(": Example tabular data resource(s) validated or invalidated against this schema."),a("br"),e._v(" "),a("em",[e._v("Oftentimes, schemas are shared with example resources to illustrate them, with valid or even invalid files (e.g. with constraint errors).")])]),e._v(" "),a("li",[a("code",[e._v("sources")]),e._v(": The source(s) used to created this schema."),a("br"),e._v(" "),a("em",[e._v("In some cases, schemas are created after a legal text or some draft specification in a human-readable document. In those cases, it’s useful to share them with the schema.")])]),e._v(" "),a("li",[a("code",[e._v("created")]),e._v(": The datetime on which this schema was created.")]),e._v(" "),a("li",[a("code",[e._v("lastModified")]),e._v(": The datetime on which this schema was last modified.")]),e._v(" "),a("li",[a("code",[e._v("version")]),e._v(": A unique version number for this schema.")]),e._v(" "),a("li",[a("code",[e._v("contributors")]),e._v(": The contributors to this schema.")])]),e._v(" "),a("h3",{attrs:{id:"example-schema"}},[a("a",{staticClass:"header-anchor",attrs:{href:"#example-schema"}},[e._v("#")]),e._v(" Example schema")]),e._v(" "),a("div",{staticClass:"language- extra-class"},[a("pre",{pre:!0,attrs:{class:"language-text"}},[a("code",[e._v('{\n "$schema": "https://specs.frictionlessdata.io/schemas/table-schema.json",\n "name": "irve",\n "title": "Infrastructures de recharge de véhicules électriques",\n "description": "Spécification du fichier d\'échange relatif aux données concernant la localisation géographique et les caractéristiques techniques des stations et des points de recharge pour véhicules électriques",\n "keywords": [\n "electric vehicle",\n "ev",\n "charging station",\n "mobility"\n ],\n "countryCode": "FR",\n "homepage": "https://github.com/etalab/schema-irve",\n "path": "https://github.com/etalab/schema-irve/raw/v1.0.1/schema.json",\n "image": "https://github.com/etalab/schema-irve/raw/v1.0.1/irve.png",\n "licenses": [\n {\n "title": "Creative Commons Zero v1.0 Universal",\n "name": "CC0-1.0",\n "path": "https://creativecommons.org/publicdomain/zero/1.0/"\n }\n ],\n "resources": [\n {\n "title": "Valid resource",\n "name": "exemple-valide",\n "path": "https://github.com/etalab/schema-irve/raw/v1.0.1/exemple-valide.csv"\n },\n {\n "title": "Invalid resource",\n "name": "exemple-invalide",\n "path": "https://github.com/etalab/schema-irve/raw/v1.0.1/exemple-invalide.csv"\n }\n ],\n "sources": [\n {\n "title": "Arrêté du 12 janvier 2017 relatif aux données concernant la localisation géographique et les caractéristiques techniques des stations et des points de recharge pour véhicules électriques",\n "path": "https://www.legifrance.gouv.fr/eli/arrete/2017/1/12/ECFI1634257A/jo/texte"\n }\n ],\n "created": "2018-06-29",\n "lastModified": "2019-05-06",\n "version": "1.0.1",\n "contributors": [\n {\n "title": "John Smith",\n "email": "john.smith@etalab.gouv.fr",\n "organization": "Etalab",\n "role": "author"\n },\n {\n "title": "Jane Doe",\n "email": "jane.doe@aol.com",\n "organization": "Civil Society Organization X",\n "role": "contributor"\n }\n ],\n "fields": [ ]\n}\n')])])]),a("h3",{attrs:{id:"implementations-7"}},[a("a",{staticClass:"header-anchor",attrs:{href:"#implementations-7"}},[e._v("#")]),e._v(" Implementations")]),e._v(" "),a("p",[e._v("The following links are actual examples already using this pattern, but not 100 % aligned with our proposal. The point is to make the Table Schema users converge towards a common pattern, before considering changing the spec.")]),e._v(" "),a("ul",[a("li",[e._v("@OpenDataFrance has initiated the creation of "),a("a",{attrs:{href:"http://git.opendatafrance.net/scdl/",target:"_blank",rel:"noopener noreferrer"}},[e._v("Table Schemas"),a("OutboundLink")],1),e._v(" to standardise common French open data datasets. "),a("a",{attrs:{href:"http://scdl.opendatafrance.net/",target:"_blank",rel:"noopener noreferrer"}},[e._v("Their Markdown documentation"),a("OutboundLink")],1),e._v(" is generated automatically from the schemas ("),a("a",{attrs:{href:"https://git.opendatafrance.net/validata/validata-doc-generator/",target:"_blank",rel:"noopener noreferrer"}},[e._v("using some scripts"),a("OutboundLink")],1),e._v("), including contextual information.")]),e._v(" "),a("li",[e._v("A tool called "),a("a",{attrs:{href:"https://go.validata.fr/",target:"_blank",rel:"noopener noreferrer"}},[e._v("Validata"),a("OutboundLink")],1),e._v(" was developed, based on Goodtables, to help French open data producers follow the schemas. It uses metadata from the schemas to present them.")]),e._v(" "),a("li",[e._v("@Etalab has launched "),a("a",{attrs:{href:"http://schema.data.gouv.fr/",target:"_blank",rel:"noopener noreferrer"}},[e._v("schema.data.gouv.fr"),a("OutboundLink")],1),e._v(", an official open data schema catalog, which is specific to France. "),a("a",{attrs:{href:"https://schema.data.gouv.fr/documentation/validation-schemas#validations-sp%C3%A9cifiques-au-format-table-schema",target:"_blank",rel:"noopener noreferrer"}},[e._v("It needs additional metadata in the schemas to validate them"),a("OutboundLink")],1),e._v(".")]),e._v(" "),a("li",[a("a",{attrs:{href:"https://github.com/etalab/schema-irve/blob/master/schema.json",target:"_blank",rel:"noopener noreferrer"}},[e._v("Example Table Schema"),a("OutboundLink")],1),e._v(" from @Etalab using metadata properties.")])]),e._v(" "),a("h2",{attrs:{id:"json-data-resources"}},[a("a",{staticClass:"header-anchor",attrs:{href:"#json-data-resources"}},[e._v("#")]),e._v(" JSON Data Resources")]),e._v(" "),a("h3",{attrs:{id:"overview-8"}},[a("a",{staticClass:"header-anchor",attrs:{href:"#overview-8"}},[e._v("#")]),e._v(" Overview")]),e._v(" "),a("p",[e._v("A simple format to describe a single structured JSON data resource. It includes support both for metadata such as author and title and a "),a("a",{attrs:{href:"https://json-schema.org/",target:"_blank",rel:"noopener noreferrer"}},[e._v("schema"),a("OutboundLink")],1),e._v(" to describe the data.")]),e._v(" "),a("h3",{attrs:{id:"introduction"}},[a("a",{staticClass:"header-anchor",attrs:{href:"#introduction"}},[e._v("#")]),e._v(" Introduction")]),e._v(" "),a("p",[e._v("A "),a("strong",[e._v("JSON Data Resource")]),e._v(" is a type of "),a("RouterLink",{attrs:{to:"/data-resource/"}},[e._v("Data Resource")]),e._v(" specialized for describing structured JSON data.")],1),e._v(" "),a("p",[e._v("JSON Data Resource extends "),a("RouterLink",{attrs:{to:"/data-resource/"}},[e._v("Data Resource")]),e._v(" in following key ways:")],1),e._v(" "),a("ul",[a("li",[e._v("The "),a("code",[e._v("schema")]),e._v(" property MUST follow the "),a("a",{attrs:{href:"https://json-schema.org/",target:"_blank",rel:"noopener noreferrer"}},[e._v("JSON Schema"),a("OutboundLink")],1),e._v(" specification,"),a("br"),e._v("\neither as a JSON object directly under the property, or a string referencing another"),a("br"),e._v("\nJSON document containing the JSON Schema")])]),e._v(" "),a("h3",{attrs:{id:"examples"}},[a("a",{staticClass:"header-anchor",attrs:{href:"#examples"}},[e._v("#")]),e._v(" Examples")]),e._v(" "),a("p",[e._v("A minimal JSON Data Resource, referencing external JSON documents, looks as follows.")]),e._v(" "),a("div",{staticClass:"language-javascript extra-class"},[a("pre",{pre:!0,attrs:{class:"language-javascript"}},[a("code",[a("span",{pre:!0,attrs:{class:"token comment"}},[e._v("// with data and a schema accessible via the local filesystem")]),e._v("\n"),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("{")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"profile"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"json-data-resource"')]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v(",")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"name"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"resource-name"')]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v(",")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"path"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("[")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"resource-path.json"')]),e._v(" "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("]")]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v(",")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"schema"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"jsonschema.json"')]),e._v("\n"),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("}")]),e._v("\n\n"),a("span",{pre:!0,attrs:{class:"token comment"}},[e._v("// with data accessible via http")]),e._v("\n"),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("{")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"profile"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"json-data-resource"')]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v(",")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"name"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"resource-name"')]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v(",")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"path"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("[")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"http://example.com/resource-path.json"')]),e._v(" "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("]")]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v(",")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"schema"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"http://example.com/jsonschema.json"')]),e._v("\n"),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("}")]),e._v("\n")])])]),a("p",[e._v("A minimal JSON Data Resource example using the data property to inline data looks as follows.")]),e._v(" "),a("div",{staticClass:"language-javascript extra-class"},[a("pre",{pre:!0,attrs:{class:"language-javascript"}},[a("code",[a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("{")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"profile"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"json-data-resource"')]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v(",")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"name"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"resource-name"')]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v(",")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"data"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("{")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"id"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token number"}},[e._v("1")]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v(",")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"first_name"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"Louise"')]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("}")]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v(",")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"schema"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("{")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"type"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"object"')]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v(",")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"required"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("[")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"id"')]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("]")]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v(",")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"properties"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("{")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"id"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("{")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"type"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"integer"')]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("}")]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v(",")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"first_name"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("{")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"type"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"string"')]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("}")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("}")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("}")]),e._v("\n"),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("}")]),e._v("\n")])])]),a("p",[e._v("A comprehensive JSON Data Resource example with all required, recommended and optional properties looks as follows.")]),e._v(" "),a("div",{staticClass:"language-javascript extra-class"},[a("pre",{pre:!0,attrs:{class:"language-javascript"}},[a("code",[a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("{")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"profile"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"json-data-resource"')]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v(",")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"name"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"solar-system"')]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v(",")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"path"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"http://example.com/solar-system.json"')]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v(",")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"title"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"The Solar System"')]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v(",")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"description"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"My favourite data about the solar system."')]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v(",")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"format"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"json"')]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v(",")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"mediatype"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"application/json"')]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v(",")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"encoding"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"utf-8"')]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v(",")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"bytes"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token number"}},[e._v("1")]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v(",")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"hash"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('""')]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v(",")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"schema"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("{")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"$schema"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"http://json-schema.org/draft-07/schema#"')]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v(",")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"type"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"object"')]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v(",")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"required"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("[")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"id"')]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("]")]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v(",")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"properties"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("{")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"id"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("{")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"type"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"integer"')]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("}")]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v(",")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"name"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("{")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"type"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"string"')]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("}")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"description"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("{")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"type"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"string"')]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("}")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("}")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("}")]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v(",")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"sources"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("[")]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("{")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"title"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"The Solar System - 2001"')]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v(",")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"path"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"http://example.com/solar-system-2001.json"')]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v(",")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"email"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('""')]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("}")]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("]")]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v(",")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"licenses"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("[")]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("{")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"name"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"CC-BY-4.0"')]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v(",")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"title"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"Creative Commons Attribution 4.0"')]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v(",")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"path"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"https://creativecommons.org/licenses/by/4.0/"')]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("}")]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("]")]),e._v("\n"),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("}")]),e._v("\n")])])]),a("h3",{attrs:{id:"specification-10"}},[a("a",{staticClass:"header-anchor",attrs:{href:"#specification-10"}},[e._v("#")]),e._v(" Specification")]),e._v(" "),a("p",[e._v("A JSON Data Resource MUST be a "),a("RouterLink",{attrs:{to:"/data-resource/"}},[e._v("Data Resource")]),e._v(", that is it MUST conform to the "),a("RouterLink",{attrs:{to:"/data-resource/"}},[e._v("Data Resource specification")]),e._v(".")],1),e._v(" "),a("p",[e._v("In addition:")]),e._v(" "),a("ul",[a("li",[e._v("The Data Resource "),a("code",[e._v("schema")]),e._v(" property MUST follow the "),a("a",{attrs:{href:"https://json-schema.org/",target:"_blank",rel:"noopener noreferrer"}},[e._v("JSON Schema"),a("OutboundLink")],1),e._v(" specification,"),a("br"),e._v("\neither as a JSON object directly under the property, or a string referencing another"),a("br"),e._v("\nJSON document containing the JSON Schema")])]),e._v(" "),a("ul",[a("li",[e._v("There "),a("code",[e._v("MUST")]),e._v(" be a "),a("code",[e._v("profile")]),e._v(" property with the value "),a("code",[e._v("json-data-resource")])])]),e._v(" "),a("ul",[a("li",[e._v("The data the Data Resource describes MUST, if non-inline, be a JSON file")])]),e._v(" "),a("h3",{attrs:{id:"json-file-requirements"}},[a("a",{staticClass:"header-anchor",attrs:{href:"#json-file-requirements"}},[e._v("#")]),e._v(" JSON file requirements")]),e._v(" "),a("p",[e._v("When "),a("code",[e._v('"format": "json"')]),e._v(", files must strictly follow the "),a("a",{attrs:{href:"https://www.json.org/",target:"_blank",rel:"noopener noreferrer"}},[e._v("JSON specification"),a("OutboundLink")],1),e._v(". Some implementations "),a("code",[e._v("MAY")]),e._v(" support "),a("code",[e._v('"format": "jsonc"')]),e._v(", allowing for non-standard single line and block comments ("),a("code",[e._v("//")]),e._v(" and "),a("code",[e._v("/* */")]),e._v(" respectively).")]),e._v(" "),a("h3",{attrs:{id:"implementations-8"}},[a("a",{staticClass:"header-anchor",attrs:{href:"#implementations-8"}},[e._v("#")]),e._v(" Implementations")]),e._v(" "),a("p",[e._v("None known.")]),e._v(" "),a("h2",{attrs:{id:"describing-data-package-catalogs-using-the-data-package-format"}},[a("a",{staticClass:"header-anchor",attrs:{href:"#describing-data-package-catalogs-using-the-data-package-format"}},[e._v("#")]),e._v(" Describing Data Package Catalogs using the Data Package Format")]),e._v(" "),a("h3",{attrs:{id:"overview-9"}},[a("a",{staticClass:"header-anchor",attrs:{href:"#overview-9"}},[e._v("#")]),e._v(" Overview")]),e._v(" "),a("p",[e._v("There are scenarios where one needs to describe a collection of data packages, such as when building an online registry, or when building a pipeline that ingests multiple datasets.")]),e._v(" "),a("p",[e._v("In these scenarios, the collection can be described using a “Catalog”, where each dataset is represented as a single resource which has:")]),e._v(" "),a("div",{staticClass:"language-json extra-class"},[a("pre",{pre:!0,attrs:{class:"language-json"}},[a("code",[a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("{")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token property"}},[e._v('"profile"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"data-package"')]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v(",")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token property"}},[e._v('"format"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"json"')]),e._v("\n"),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("}")]),e._v("\n")])])]),a("h3",{attrs:{id:"specification-11"}},[a("a",{staticClass:"header-anchor",attrs:{href:"#specification-11"}},[e._v("#")]),e._v(" Specification")]),e._v(" "),a("p",[e._v("The Data Package Catalog builds directly on the Data Package specification. Thus a Data Package Catalog "),a("code",[e._v("MUST")]),e._v(" be a Data Package and conform to the "),a("RouterLink",{attrs:{to:"/data-package/"}},[e._v("Data Package specification")]),e._v(".")],1),e._v(" "),a("p",[e._v("The Data Package Catalog has the following requirements over and above those imposed by Data Package:")]),e._v(" "),a("ul",[a("li",[e._v("There "),a("code",[e._v("MUST")]),e._v(" be a "),a("code",[e._v("profile")]),e._v(" property with the value "),a("code",[e._v("data-package-catalog")]),e._v(", or a "),a("code",[e._v("profile")]),e._v(" that extends it")]),e._v(" "),a("li",[e._v("Each resource "),a("code",[e._v("MUST")]),e._v(" also be a Data Package")])]),e._v(" "),a("h4",{attrs:{id:"examples-2"}},[a("a",{staticClass:"header-anchor",attrs:{href:"#examples-2"}},[e._v("#")]),e._v(" Examples")]),e._v(" "),a("p",[e._v("A generic package catalog:")]),e._v(" "),a("div",{staticClass:"language-json extra-class"},[a("pre",{pre:!0,attrs:{class:"language-json"}},[a("code",[a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("{")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token property"}},[e._v('"profile"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"data-package-catalog"')]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v(",")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token property"}},[e._v('"name"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"climate-change-packages"')]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v(",")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token property"}},[e._v('"resources"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("[")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("{")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token property"}},[e._v('"profile"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"json-data-package"')]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v(",")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token property"}},[e._v('"format"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"json"')]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v(",")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token property"}},[e._v('"name"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"beacon-network-description"')]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v(",")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token property"}},[e._v('"path"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"https://http://beacon.berkeley.edu/hypothetical_deployment_description.json"')]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("}")]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v(",")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("{")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token property"}},[e._v('"profile"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"tabular-data-package"')]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v(",")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token property"}},[e._v('"format"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"json"')]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v(",")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token property"}},[e._v('"path"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"https://pkgstore.datahub.io/core/co2-ppm/10/datapackage.json"')]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("}")]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v(",")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("{")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token property"}},[e._v('"profile"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"tabular-data-package"')]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v(",")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token property"}},[e._v('"name"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"co2-fossil-global"')]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v(",")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token property"}},[e._v('"format"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"json"')]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v(",")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token property"}},[e._v('"path"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"https://pkgstore.datahub.io/core/co2-fossil-global/11/datapackage.json"')]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("}")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("]")]),e._v("\n"),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("}")]),e._v("\n")])])]),a("p",[e._v("A minimal tabular data catalog:")]),e._v(" "),a("div",{staticClass:"language-json extra-class"},[a("pre",{pre:!0,attrs:{class:"language-json"}},[a("code",[a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("{")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token property"}},[e._v('"profile"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"tabular-data-package-catalog"')]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v(",")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token property"}},[e._v('"name"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"datahub-climate-change-packages"')]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v(",")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token property"}},[e._v('"resources"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("[")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("{")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token property"}},[e._v('"path"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"https://pkgstore.datahub.io/core/co2-ppm/10/datapackage.json"')]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("}")]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v(",")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("{")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token property"}},[e._v('"name"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"co2-fossil-global"')]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v(",")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token property"}},[e._v('"path"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"https://pkgstore.datahub.io/core/co2-fossil-global/11/datapackage.json"')]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("}")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("]")]),e._v("\n"),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("}")]),e._v("\n")])])]),a("p",[e._v("Data packages can also be declared inline in the data catalog:")]),e._v(" "),a("div",{staticClass:"language-json extra-class"},[a("pre",{pre:!0,attrs:{class:"language-json"}},[a("code",[a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("{")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token property"}},[e._v('"profile"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"tabular-data-package-catalog"')]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v(",")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token property"}},[e._v('"name"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"my-data-catalog"')]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v(",")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token property"}},[e._v('"resources"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("[")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("{")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token property"}},[e._v('"profile"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"tabular-data-package"')]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v(",")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token property"}},[e._v('"name"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"my-dataset"')]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v(",")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token comment"}},[e._v("// here we list the data files in this dataset")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token property"}},[e._v('"resources"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("[")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("{")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token property"}},[e._v('"profile"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"tabular-data-resource"')]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v(",")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token property"}},[e._v('"name"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"resource-name"')]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v(",")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token property"}},[e._v('"data"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("[")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("{")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token property"}},[e._v('"id"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token number"}},[e._v("1")]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v(",")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token property"}},[e._v('"first_name"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"Louise"')]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("}")]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v(",")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("{")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token property"}},[e._v('"id"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token number"}},[e._v("2")]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v(",")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token property"}},[e._v('"first_name"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"Julia"')]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("}")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("]")]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v(",")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token property"}},[e._v('"schema"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("{")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token property"}},[e._v('"fields"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("[")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("{")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token property"}},[e._v('"name"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"id"')]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v(",")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token property"}},[e._v('"type"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"integer"')]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("}")]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v(",")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("{")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token property"}},[e._v('"name"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"first_name"')]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v(",")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token property"}},[e._v('"type"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"string"')]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("}")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("]")]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v(",")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token property"}},[e._v('"primaryKey"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"id"')]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("}")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("}")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("]")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("}")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("]")]),e._v("\n"),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("}")]),e._v("\n")])])]),a("h3",{attrs:{id:"implementations-9"}},[a("a",{staticClass:"header-anchor",attrs:{href:"#implementations-9"}},[e._v("#")]),e._v(" Implementations")]),e._v(" "),a("p",[e._v("None known.")]),e._v(" "),a("h2",{attrs:{id:"table-schema-unique-constraints"}},[a("a",{staticClass:"header-anchor",attrs:{href:"#table-schema-unique-constraints"}},[e._v("#")]),e._v(" Table Schema: Unique constraints")]),e._v(" "),a("h3",{attrs:{id:"overview-10"}},[a("a",{staticClass:"header-anchor",attrs:{href:"#overview-10"}},[e._v("#")]),e._v(" Overview")]),e._v(" "),a("p",[e._v("A "),a("code",[e._v("primaryKey")]),e._v(" uniquely identifies each row in a table. Per SQL standards, it"),a("br"),e._v("\ncannot contain "),a("code",[e._v("null")]),e._v(" values. This pattern implements the SQL UNIQUE constraint"),a("br"),e._v("\nby introducing a "),a("code",[e._v("uniqueKeys")]),e._v(" array, defining one or more row uniqueness"),a("br"),e._v("\nconstraints which do support "),a("code",[e._v("null")]),e._v(" values. An additional "),a("code",[e._v("uniqueNulls")]),e._v(" property"),a("br"),e._v("\ncontrols how "),a("code",[e._v("null")]),e._v(" values are to be treated in unique constraints.")]),e._v(" "),a("h3",{attrs:{id:"specification-12"}},[a("a",{staticClass:"header-anchor",attrs:{href:"#specification-12"}},[e._v("#")]),e._v(" Specification")]),e._v(" "),a("h4",{attrs:{id:"uniquekeys-add"}},[a("a",{staticClass:"header-anchor",attrs:{href:"#uniquekeys-add"}},[e._v("#")]),e._v(" "),a("code",[e._v("uniqueKeys")]),e._v(" (add)")]),e._v(" "),a("p",[e._v("The "),a("code",[e._v("uniqueKeys")]),e._v(" property, if present, "),a("code",[e._v("MUST")]),e._v(" be an array. Each entry"),a("br"),e._v("\n("),a("code",[e._v("uniqueKey")]),e._v(") in the array "),a("code",[e._v("MUST")]),e._v(" be a string or array (structured as per"),a("br"),e._v(" "),a("code",[e._v("primaryKey")]),e._v(") specifying the resource field or fields required to be unique for"),a("br"),e._v("\neach row in the table.")]),e._v(" "),a("h4",{attrs:{id:"uniquenulls-add"}},[a("a",{staticClass:"header-anchor",attrs:{href:"#uniquenulls-add"}},[e._v("#")]),e._v(" "),a("code",[e._v("uniqueNulls")]),e._v(" (add)")]),e._v(" "),a("p",[e._v("The "),a("code",[e._v("uniqueNulls")]),e._v(" property is a boolean that dictates how "),a("code",[e._v("null")]),e._v(" values should"),a("br"),e._v("\nbe treated by all unique constraints set on a resource.")]),e._v(" "),a("ul",[a("li",[e._v("If "),a("code",[e._v("true")]),e._v(" (the default), "),a("code",[e._v("null")]),e._v(" values are treated as unique (per most SQL"),a("br"),e._v("\ndatabases). By this definition, "),a("code",[e._v("1, null, null")]),e._v(" is UNIQUE.")]),e._v(" "),a("li",[e._v("If "),a("code",[e._v("false")]),e._v(", "),a("code",[e._v("null")]),e._v(" values are treated like any other value (per Microsoft SQL"),a("br"),e._v("\nServer, Python pandas, R data.frame, Google Sheets). By this definition, "),a("code",[e._v("1, null, null")]),e._v(" is NOT UNIQUE.")])]),e._v(" "),a("h4",{attrs:{id:"foreignkeys-edit"}},[a("a",{staticClass:"header-anchor",attrs:{href:"#foreignkeys-edit"}},[e._v("#")]),e._v(" "),a("code",[e._v("foreignKeys")]),e._v(" (edit)")]),e._v(" "),a("p",[e._v("Per SQL standards, "),a("code",[e._v("null")]),e._v(" values are permitted in both the local and reference"),a("br"),e._v("\nkeys of a foreign key. However, reference keys "),a("code",[e._v("MUST")]),e._v(" be unique and are"),a("br"),e._v("\ntherefore equivalent to a "),a("code",[e._v("uniqueKey")]),e._v(" set on the reference resource (the meaning"),a("br"),e._v("\nof which is determined by the reference "),a("code",[e._v("uniqueNulls")]),e._v(").")]),e._v(" "),a("p",[e._v("Furthermore, per SQL standards, the local key "),a("code",[e._v("MAY")]),e._v(" contain keys with field"),a("br"),e._v("\nvalues not present in the reference key if and only if at least one of the"),a("br"),e._v("\nfields is locally "),a("code",[e._v("null")]),e._v(". For example, "),a("code",[e._v("(1, null)")]),e._v(" is permitted locally even if"),a("br"),e._v("\nthe reference is "),a("code",[e._v("[(2, 1), (3, 1)]")]),e._v(". This behavior is the same regardless of the"),a("br"),e._v("\nvalue of "),a("code",[e._v("uniqueNulls")]),e._v(".")]),e._v(" "),a("h3",{attrs:{id:"examples-3"}},[a("a",{staticClass:"header-anchor",attrs:{href:"#examples-3"}},[e._v("#")]),e._v(" Examples")]),e._v(" "),a("h4",{attrs:{id:"null-in-unique-constraints"}},[a("a",{staticClass:"header-anchor",attrs:{href:"#null-in-unique-constraints"}},[e._v("#")]),e._v(" "),a("code",[e._v("null")]),e._v(" in unique constraints")]),e._v(" "),a("table",[a("thead",[a("tr",[a("th",[e._v("a")]),e._v(" "),a("th",[e._v("b")]),e._v(" "),a("th",[e._v("c")]),e._v(" "),a("th",[e._v("d")])])]),e._v(" "),a("tbody",[a("tr",[a("td",[e._v("1")]),e._v(" "),a("td",[e._v("1")]),e._v(" "),a("td",[e._v("1")]),e._v(" "),a("td",[e._v("1")])]),e._v(" "),a("tr",[a("td",[e._v("2")]),e._v(" "),a("td",[e._v("2")]),e._v(" "),a("td",[a("code",[e._v("null")])]),e._v(" "),a("td",[e._v("2")])]),e._v(" "),a("tr",[a("td",[e._v("3")]),e._v(" "),a("td",[e._v("2")]),e._v(" "),a("td",[a("code",[e._v("null")])]),e._v(" "),a("td",[a("code",[e._v("null")])])])])]),e._v(" "),a("p",[e._v("The above table meets the following primary key and two unique key constraints:")]),e._v(" "),a("div",{staticClass:"language-json extra-class"},[a("pre",{pre:!0,attrs:{class:"language-json"}},[a("code",[a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("{")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token property"}},[e._v('"primaryKey"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("[")]),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"a"')]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("]")]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v(",")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token property"}},[e._v('"uniqueKeys"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("[")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("[")]),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"b"')]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v(",")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"c"')]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("]")]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v(",")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("[")]),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"c"')]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v(",")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"d"')]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("]")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("]")]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v(",")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token property"}},[e._v('"uniqueNulls"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token boolean"}},[e._v("true")]),e._v("\n"),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("}")]),e._v("\n")])])]),a("p",[e._v("The primary key "),a("code",[e._v("(a)")]),e._v(" only contains unique, non-"),a("code",[e._v("null")]),e._v(" values. In contrast, the"),a("br"),e._v("\nunique keys can contain "),a("code",[e._v("null")]),e._v(" values. Although unique key "),a("code",[e._v("(b, c)")]),e._v(" contains two"),a("br"),e._v("\nidentical keys "),a("code",[e._v("(2, null)")]),e._v(", this is permitted because "),a("code",[e._v("uniqueNulls: true")]),a("br"),e._v("\nspecifies that "),a("code",[e._v("null")]),e._v(" values are unique. This behavior is consistent with the"),a("br"),e._v("\nUNIQUE constraint of PostgreSQL and most other SQL implementations, as"),a("br"),e._v("\nillustrated by this"),a("br"),e._v(" "),a("a",{attrs:{href:"https://dbfiddle.uk/?rdbms=postgres_11&fiddle=34cab8ba7d74b488d215a96f7e83c096",target:"_blank",rel:"noopener noreferrer"}},[e._v("dbfiddle"),a("OutboundLink")],1),e._v("."),a("br"),e._v("\nThe same keys would be considered duplicates if "),a("code",[e._v("uniqueNulls: false")]),e._v(", consistent"),a("br"),e._v("\nwith the UNIQUE constraint of Microsoft SQL Server, as illustrated by this"),a("br"),e._v(" "),a("a",{attrs:{href:"https://dbfiddle.uk/?rdbms=sqlserver_2019l&fiddle=34cab8ba7d74b488d215a96f7e83c096",target:"_blank",rel:"noopener noreferrer"}},[e._v("dbfiddle"),a("OutboundLink")],1),e._v(".")]),e._v(" "),a("h4",{attrs:{id:"setting-unique-constraints"}},[a("a",{staticClass:"header-anchor",attrs:{href:"#setting-unique-constraints"}},[e._v("#")]),e._v(" Setting unique constraints")]),e._v(" "),a("p",[e._v("For a given resource, unique constraints can be set for one field using a"),a("br"),e._v("\nfield’s "),a("code",[e._v("unique")]),e._v(" constraint, for one or multiple fields using a "),a("code",[e._v("uniqueKey")]),e._v(", and"),a("br"),e._v("\nfor one or multiple fields using a "),a("code",[e._v("foreignKey")]),e._v(" referencing the resource. Each"),a("br"),e._v("\nof the following examples set a unique constraint on field "),a("code",[e._v("a")]),e._v(":")]),e._v(" "),a("p",[a("strong",[e._v("Field "),a("code",[e._v("constraints")])])]),e._v(" "),a("div",{staticClass:"language-json extra-class"},[a("pre",{pre:!0,attrs:{class:"language-json"}},[a("code",[a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("{")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token property"}},[e._v('"fields"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("[")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("{")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token property"}},[e._v('"name"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"a"')]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v(",")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token property"}},[e._v('"constraints"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("{")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token property"}},[e._v('"unique"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token boolean"}},[e._v("true")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("}")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("}")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("]")]),e._v("\n"),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("}")]),e._v("\n")])])]),a("p",[a("strong",[a("code",[e._v("uniqueKeys")])])]),e._v(" "),a("div",{staticClass:"language-json extra-class"},[a("pre",{pre:!0,attrs:{class:"language-json"}},[a("code",[a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("{")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token property"}},[e._v('"uniqueKeys"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("[")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"a"')]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("]")]),e._v("\n"),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("}")]),e._v("\n")])])]),a("p",[a("strong",[a("code",[e._v("foreignKeys")])])]),e._v(" "),a("div",{staticClass:"language-json extra-class"},[a("pre",{pre:!0,attrs:{class:"language-json"}},[a("code",[a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("{")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token property"}},[e._v('"foreignKeys"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("[")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("{")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token property"}},[e._v('"fields"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"a"')]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v(",")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token property"}},[e._v('"reference"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("{")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token property"}},[e._v('"resource"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('""')]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v(",")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token property"}},[e._v('"fields"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"a"')]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("}")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("}")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("]")]),e._v("\n"),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("}")]),e._v("\n")])])]),a("h3",{attrs:{id:"implementations-10"}},[a("a",{staticClass:"header-anchor",attrs:{href:"#implementations-10"}},[e._v("#")]),e._v(" Implementations")]),e._v(" "),a("p",[e._v("None known.")]),e._v(" "),a("h2",{attrs:{id:"describing-files-inside-a-compressed-file-such-as-zip"}},[a("a",{staticClass:"header-anchor",attrs:{href:"#describing-files-inside-a-compressed-file-such-as-zip"}},[e._v("#")]),e._v(" Describing files inside a compressed file such as Zip")]),e._v(" "),a("h3",{attrs:{id:"overview-11"}},[a("a",{staticClass:"header-anchor",attrs:{href:"#overview-11"}},[e._v("#")]),e._v(" Overview")]),e._v(" "),a("p",[e._v("Some datasets need to contain a Zip file (or tar, other formats) containing a"),a("br"),e._v("\nset of files.")]),e._v(" "),a("p",[e._v("This might happen for practical reasons (datasets containing thousands of files)"),a("br"),e._v("\nor for technical limitations (for example, currently Zenodo doesn’t support subdirectories and"),a("br"),e._v("\ndatasets might need subdirectory structures to be useful).")]),e._v(" "),a("h3",{attrs:{id:"implementations-11"}},[a("a",{staticClass:"header-anchor",attrs:{href:"#implementations-11"}},[e._v("#")]),e._v(" Implementations")]),e._v(" "),a("p",[e._v("There are no known implementations at present.")]),e._v(" "),a("h3",{attrs:{id:"specification-13"}},[a("a",{staticClass:"header-anchor",attrs:{href:"#specification-13"}},[e._v("#")]),e._v(" Specification")]),e._v(" "),a("p",[e._v("The "),a("code",[e._v("resources")]),e._v(" in a "),a("code",[e._v("data-package")]),e._v(" can contain “recursive resources”: identifying"),a("br"),e._v("\na new resource.")]),e._v(" "),a("h3",{attrs:{id:"example"}},[a("a",{staticClass:"header-anchor",attrs:{href:"#example"}},[e._v("#")]),e._v(" Example")]),e._v(" "),a("div",{staticClass:"language-json extra-class"},[a("pre",{pre:!0,attrs:{class:"language-json"}},[a("code",[a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("{")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token property"}},[e._v('"profile"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"data-package"')]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v(",")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token property"}},[e._v('"resources"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("[")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("{")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token property"}},[e._v('"path"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"https://zenodo.org/record/3247384/files/Sea-Bird_Processed_Data.zip"')]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v(",")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token property"}},[e._v('"format"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"zip"')]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v(",")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token property"}},[e._v('"mediatype"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"application/zip"')]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v(",")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token property"}},[e._v('"bytes"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"294294242424"')]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v(",")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token property"}},[e._v('"hash"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"a27063c614c183b502e5c03bd9c8931b"')]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v(",")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token property"}},[e._v('"resources"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("[")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("{")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token property"}},[e._v('"path"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"file_name.csv"')]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v(",")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token property"}},[e._v('"format"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"csv"')]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v(",")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token property"}},[e._v('"mediatype"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"text/csv"')]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v(",")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token property"}},[e._v('"bytes"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token number"}},[e._v("242421")]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v(",")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token property"}},[e._v('"hash"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"0300048878bb9b5804a1f62869d296bc"')]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v(",")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token property"}},[e._v('"profile"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"tabular-data-resource"')]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v(",")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token property"}},[e._v('"schema"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"tableschema.json"')]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("}")]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v(",")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("{")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token property"}},[e._v('"path"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"directory/file_name2.csv"')]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v(",")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token property"}},[e._v('"format"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"csv"')]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v(",")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token property"}},[e._v('"mediatype"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"text/csv"')]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v(",")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token property"}},[e._v('"bytes"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token number"}},[e._v("2424213")]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v(",")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token property"}},[e._v('"hash"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"ff9435e0ee350efbe8a4a8779a47caaa"')]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v(",")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token property"}},[e._v('"profile"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"tabular-data-resource"')]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v(",")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token property"}},[e._v('"schema"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"tableschema.json"')]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("}")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("]")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("}")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("]")]),e._v("\n"),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("}")]),e._v("\n")])])]),a("p",[e._v("For a "),a("code",[e._v(".tar.gz")]),e._v(" it would be the same changing the "),a("code",[e._v('"format"')]),e._v(" and the"),a("br"),e._v(" "),a("code",[e._v('"mediatype"')]),e._v(".")]),e._v(" "),a("h3",{attrs:{id:"types-of-files"}},[a("a",{staticClass:"header-anchor",attrs:{href:"#types-of-files"}},[e._v("#")]),e._v(" Types of files")]),e._v(" "),a("p",[e._v("Support for "),a("code",[e._v("Zip")]),e._v(" and "),a("code",[e._v("tar.gz")]),e._v(" might be enough: hopefully everything can be"),a("br"),e._v("\nre-packaged using these formats.")]),e._v(" "),a("p",[e._v("To keep the implementation and testing testing: only one recursive level is"),a("br"),e._v("\npossible. A "),a("code",[e._v("resource")]),e._v(" can list "),a("code",[e._v("resources")]),e._v(" inside (like in the example). But"),a("br"),e._v("\nthe inner resources cannot contain resources again.")]),e._v(" "),a("h2",{attrs:{id:"missing-values-per-field"}},[a("a",{staticClass:"header-anchor",attrs:{href:"#missing-values-per-field"}},[e._v("#")]),e._v(" Missing values per field")]),e._v(" "),a("h3",{attrs:{id:"overview-12"}},[a("a",{staticClass:"header-anchor",attrs:{href:"#overview-12"}},[e._v("#")]),e._v(" Overview")]),e._v(" "),a("p",[e._v("Characters representing missing values in a table can be defined for all fields in a "),a("a",{attrs:{href:"http://frictionlessdata.io/specs/tabular-data-resource/",target:"_blank",rel:"noopener noreferrer"}},[e._v("Tabular Data Resource"),a("OutboundLink")],1),e._v(" using the "),a("a",{attrs:{href:"http://frictionlessdata.io/specs/table-schema/#missing-values",target:"_blank",rel:"noopener noreferrer"}},[a("code",[e._v("missingValues")]),a("OutboundLink")],1),e._v(" property in a Table Schema. Values that match the "),a("code",[e._v("missingValues")]),e._v(" are treated as "),a("code",[e._v("null")]),e._v(".")]),e._v(" "),a("p",[e._v("The Missing values per field pattern allows different missing values to be specified for each field in a Table Schema. If not specified, each field inherits from values assigned to "),a("code",[e._v("missingValues")]),e._v(" at the Tabular Data Resource level.")]),e._v(" "),a("p",[e._v("For example, this data…")]),e._v(" "),a("table",[a("thead",[a("tr",[a("th",[e._v("item")]),e._v(" "),a("th",[e._v("description")]),e._v(" "),a("th",[e._v("price")])])]),e._v(" "),a("tbody",[a("tr",[a("td",[e._v("1")]),e._v(" "),a("td",[e._v("Apple")]),e._v(" "),a("td",[e._v("0.99")])]),e._v(" "),a("tr",[a("td",[e._v("tba")]),e._v(" "),a("td",[e._v("Banana")]),e._v(" "),a("td",[e._v("-1")])]),e._v(" "),a("tr",[a("td",[e._v("3")]),e._v(" "),a("td",[e._v("n/a")]),e._v(" "),a("td",[e._v("1.20")])])])]),e._v(" "),a("p",[e._v("…using this Table Schema…")]),e._v(" "),a("div",{staticClass:"language-javascript extra-class"},[a("pre",{pre:!0,attrs:{class:"language-javascript"}},[a("code",[a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"schema"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("{")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"fields"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("[")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("{")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"name"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"item"')]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v(",")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"title"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"An inventory item number"')]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v(",")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"type"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"integer"')]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("}")]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v(",")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("{")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"name"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"description"')]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v(",")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"title"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"item description"')]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v(",")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"type"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"string"')]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v(",")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"missingValues"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("[")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"n/a"')]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("]")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("}")]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v(",")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("{")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"name"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"price"')]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v(",")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"title"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"cost price"')]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v(",")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"type"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"number"')]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v(",")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"missingValues"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("[")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"-1"')]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("]")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("}")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("]")]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v(",")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"missingValues"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("[")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"tba"')]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v(",")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('""')]),e._v(" "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("]")]),e._v("\n"),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("}")]),e._v("\n")])])]),a("p",[e._v("…would be interpreted as…")]),e._v(" "),a("table",[a("thead",[a("tr",[a("th",[e._v("item")]),e._v(" "),a("th",[e._v("description")]),e._v(" "),a("th",[e._v("price")])])]),e._v(" "),a("tbody",[a("tr",[a("td",[e._v("1")]),e._v(" "),a("td",[e._v("Apple")]),e._v(" "),a("td",[e._v("0.99")])]),e._v(" "),a("tr",[a("td",[a("code",[e._v("null")])]),e._v(" "),a("td",[e._v("Banana")]),e._v(" "),a("td",[a("code",[e._v("null")])])]),e._v(" "),a("tr",[a("td",[e._v("3")]),e._v(" "),a("td",[a("code",[e._v("null")])]),e._v(" "),a("td",[e._v("1.20")])])])]),e._v(" "),a("h3",{attrs:{id:"specification-14"}},[a("a",{staticClass:"header-anchor",attrs:{href:"#specification-14"}},[e._v("#")]),e._v(" Specification")]),e._v(" "),a("p",[e._v("A field MAY have a "),a("code",[e._v("missingValues")]),e._v(" property that MUST be an "),a("code",[e._v("array")]),e._v(" where each entry is a "),a("code",[e._v("string")]),e._v(". If not specified, each field inherits from the values assigned to "),a("a",{attrs:{href:"http://frictionlessdata.io/specs/table-schema/#missing-values",target:"_blank",rel:"noopener noreferrer"}},[a("code",[e._v("missingValues")]),a("OutboundLink")],1),e._v(" at the Tabular Data Resource level.")]),e._v(" "),a("h3",{attrs:{id:"implementations-12"}},[a("a",{staticClass:"header-anchor",attrs:{href:"#implementations-12"}},[e._v("#")]),e._v(" Implementations")]),e._v(" "),a("p",[e._v("None known.")]),e._v(" "),a("h2",{attrs:{id:"table-schema-enum-labels-and-ordering"}},[a("a",{staticClass:"header-anchor",attrs:{href:"#table-schema-enum-labels-and-ordering"}},[e._v("#")]),e._v(" Table Schema: Enum labels and ordering")]),e._v(" "),a("h3",{attrs:{id:"overview-13"}},[a("a",{staticClass:"header-anchor",attrs:{href:"#overview-13"}},[e._v("#")]),e._v(" Overview")]),e._v(" "),a("p",[e._v("Many software packages for manipulating and analyzing tabular data have special"),a("br"),e._v("\nfeatures for working with categorical variables. These include:")]),e._v(" "),a("ul",[a("li",[e._v("Value labels or formats ("),a("a",{attrs:{href:"https://www.stata.com/manuals13/dlabel.pdf",target:"_blank",rel:"noopener noreferrer"}},[e._v("Stata"),a("OutboundLink")],1),e._v(","),a("br"),e._v(" "),a("a",{attrs:{href:"https://documentation.sas.com/doc/en/pgmsascdc/9.4_3.5/proc/p1upn25lbfo6mkn1wncu4dyh9q91.htm",target:"_blank",rel:"noopener noreferrer"}},[e._v("SAS"),a("OutboundLink")],1),a("br"),e._v("\nand "),a("a",{attrs:{href:"https://www.ibm.com/docs/en/spss-statistics/beta?topic=data-adding-value-labels",target:"_blank",rel:"noopener noreferrer"}},[e._v("SPSS"),a("OutboundLink")],1),e._v(")")]),e._v(" "),a("li",[a("a",{attrs:{href:"https://pandas.pydata.org/docs/user_guide/categorical.html",target:"_blank",rel:"noopener noreferrer"}},[e._v("Categoricals (Pandas)"),a("OutboundLink")],1)]),e._v(" "),a("li",[a("a",{attrs:{href:"https://www.stat.berkeley.edu/~s133/factors.html",target:"_blank",rel:"noopener noreferrer"}},[e._v("Factors ®"),a("OutboundLink")],1)]),e._v(" "),a("li",[a("a",{attrs:{href:"https://dataframes.juliadata.org/stable/man/categorical/",target:"_blank",rel:"noopener noreferrer"}},[e._v("CategoricalVectors (Julia)"),a("OutboundLink")],1)])]),e._v(" "),a("p",[e._v("These features can result in more efficient storage and faster runtime"),a("br"),e._v("\nperformance, but more importantly, facilitate analysis by indicating that a"),a("br"),e._v("\nvariable should be treated as categorical and by permitting the logical order"),a("br"),e._v("\nof the categories to differ from their lexical order. And in the case of value"),a("br"),e._v("\nlabels, they permit the analyst to work with variables in numeric form (e.g.,"),a("br"),e._v("\nin expressions, when fitting models) while generating output (e.g., tables,"),a("br"),e._v("\nplots) that is labeled with informative strings.")]),e._v(" "),a("p",[e._v("While these features are of limited use in some disciplines, others rely"),a("br"),e._v("\nheavily on them (e.g., social sciences, epidemiology, clinical research,"),a("br"),e._v("\netc.). Thus, before these disciplines can begin to use Frictionless in a"),a("br"),e._v("\nmeaningful way, both the standards and the software tools need to support"),a("br"),e._v("\nthese features. This pattern addresses necessary extensions to the"),a("br"),e._v(" "),a("a",{attrs:{href:"https://specs.frictionlessdata.io//table-schema/",target:"_blank",rel:"noopener noreferrer"}},[e._v("Table Schema"),a("OutboundLink")],1),e._v(".")]),e._v(" "),a("h3",{attrs:{id:"principles"}},[a("a",{staticClass:"header-anchor",attrs:{href:"#principles"}},[e._v("#")]),e._v(" Principles")]),e._v(" "),a("p",[e._v("Before describing the proposed extensions, here are the principles on which"),a("br"),e._v("\nthey are based:")]),e._v(" "),a("ol",[a("li",[e._v("Extensions should be software agnostic (i.e., no additions to the official"),a("br"),e._v("\nschema targeted toward a specific piece of software). While the extensions"),a("br"),e._v("\nare intended to support the use of features not available in all software,"),a("br"),e._v("\nthe resulting data package should continue to work as well as possible with"),a("br"),e._v("\nsoftware that does not have those features.")]),e._v(" "),a("li",[e._v("Related to (1), extensions should only include metadata that describe the"),a("br"),e._v("\ndata themselves—not instructions for what a specific software package should"),a("br"),e._v("\ndo with the data. Users who want to include the latter may do so within"),a("br"),e._v("\na sub-namespace such as "),a("code",[e._v("custom")]),e._v(" (e.g., see Issues "),a("a",{attrs:{href:"https://github.com/frictionlessdata/specs/issues/103",target:"_blank",rel:"noopener noreferrer"}},[e._v("#103"),a("OutboundLink")],1),a("br"),e._v("\nand "),a("a",{attrs:{href:"https://github.com/frictionlessdata/specs/issues/663",target:"_blank",rel:"noopener noreferrer"}},[e._v("#663"),a("OutboundLink")],1),e._v(").")]),e._v(" "),a("li",[e._v("Extensions must be backward compatible (i.e., not break existing tools,"),a("br"),e._v("\nworkflows, etc. for working with Frictionless packages).")])]),e._v(" "),a("p",[e._v("It is worth emphasizing that the scope of the proposed extensions is strictly"),a("br"),e._v("\nlimited to the information necessary to make use of the features for working"),a("br"),e._v("\nwith categorical data provided by the software packages listed above. Previous"),a("br"),e._v("\ndiscussions of this issue have occasionally included references to additional"),a("br"),e._v("\nvariable-level metadata (e.g., multiple sets of category labels such as both"),a("br"),e._v("\n“short labels” and longer “descriptions”, or links to common data elements,"),a("br"),e._v("\ncontrolled vocabularies or rdfTypes). While these additional metadata are"),a("br"),e._v("\nundoubtedly useful, we speculate that the large majority of users who would"),a("br"),e._v("\nbenefit from the extensions propopsed here would not have and/or utilize such"),a("br"),e._v("\ninformation, and therefore argue that these should be considered under a"),a("br"),e._v("\nseparate proposal.")]),e._v(" "),a("h3",{attrs:{id:"implementations-13"}},[a("a",{staticClass:"header-anchor",attrs:{href:"#implementations-13"}},[e._v("#")]),e._v(" Implementations")]),e._v(" "),a("p",[e._v("Our proposal to add a field-specific "),a("code",[e._v("enumOrdered")]),e._v(" property has been raised"),a("br"),e._v(" "),a("a",{attrs:{href:"https://github.com/frictionlessdata/specs/issues/739",target:"_blank",rel:"noopener noreferrer"}},[e._v("here"),a("OutboundLink")],1),e._v(" and"),a("br"),e._v(" "),a("a",{attrs:{href:"https://github.com/frictionlessdata/specs/issues/156",target:"_blank",rel:"noopener noreferrer"}},[e._v("here"),a("OutboundLink")],1),e._v(".")]),e._v(" "),a("p",[e._v("Discussions regarding supporting software providing features for working with"),a("br"),e._v("\ncategorical variables appear in the following GitHub issues:")]),e._v(" "),a("ul",[a("li",[a("a",{attrs:{href:"https://github.com/frictionlessdata/specs/issues/156",target:"_blank",rel:"noopener noreferrer"}},[e._v("https://github.com/frictionlessdata/specs/issues/156"),a("OutboundLink")],1)]),e._v(" "),a("li",[a("a",{attrs:{href:"https://github.com/frictionlessdata/specs/issues/739",target:"_blank",rel:"noopener noreferrer"}},[e._v("https://github.com/frictionlessdata/specs/issues/739"),a("OutboundLink")],1)])]),e._v(" "),a("p",[e._v("and in the Frictionless Data forum:")]),e._v(" "),a("ul",[a("li",[a("a",{attrs:{href:"https://discuss.okfn.org/t/can-you-add-code-descriptions-to-a-data-package/",target:"_blank",rel:"noopener noreferrer"}},[e._v("https://discuss.okfn.org/t/can-you-add-code-descriptions-to-a-data-package/"),a("OutboundLink")],1)]),e._v(" "),a("li",[a("a",{attrs:{href:"https://discuss.okfn.org/t/something-like-rs-ordered-factors-or-enums-as-column-type/",target:"_blank",rel:"noopener noreferrer"}},[e._v("https://discuss.okfn.org/t/something-like-rs-ordered-factors-or-enums-as-column-type/"),a("OutboundLink")],1)])]),e._v(" "),a("p",[e._v("Finally, while we are unaware of any existing implementations intended for"),a("br"),e._v("\ngeneral use, it is likely that many users are already exploiting the fact that"),a("br"),e._v("\narbitrary fields may be added to the"),a("br"),e._v(" "),a("a",{attrs:{href:"https://specs.frictionlessdata.io//table-schema/",target:"_blank",rel:"noopener noreferrer"}},[e._v("table schema"),a("OutboundLink")],1),a("br"),e._v("\nto support internal implementations.")]),e._v(" "),a("h3",{attrs:{id:"proposed-extensions"}},[a("a",{staticClass:"header-anchor",attrs:{href:"#proposed-extensions"}},[e._v("#")]),e._v(" Proposed extensions")]),e._v(" "),a("p",[e._v("We propose two extensions to "),a("a",{attrs:{href:"https://specs.frictionlessdata.io/table-schema/",target:"_blank",rel:"noopener noreferrer"}},[e._v("Table Schema"),a("OutboundLink")],1),e._v(":")]),e._v(" "),a("ol",[a("li",[e._v("Add an optional field-specific "),a("code",[e._v("enumOrdered")]),e._v(" property, which can be used"),a("br"),e._v("\nwhen contructing a categorical (or factor) to indicate that the variable is"),a("br"),e._v("\nordinal.")]),e._v(" "),a("li",[e._v("Add an optional field-specific "),a("code",[e._v("enumLabels")]),e._v(" property for use when data are"),a("br"),e._v("\nstored using integer or other codes rather than using the category labels."),a("br"),e._v("\nThis contains an object mapping the codes appearing in the data (keys) to"),a("br"),e._v("\nwhat they mean (values), and can be used by software to construct"),a("br"),e._v("\ncorresponding value labels or categoricals (when supported) or to translate"),a("br"),e._v("\nthe values when reading the data.")])]),e._v(" "),a("p",[e._v("These extensions are fully backward compatible, since they are optional and"),a("br"),e._v("\nnot providing them is valid.")]),e._v(" "),a("p",[e._v("Here is an example of a categorical variable using extension (1):")]),e._v(" "),a("div",{staticClass:"language- extra-class"},[a("pre",{pre:!0,attrs:{class:"language-text"}},[a("code",[e._v('{\n "fields": [\n {\n "name": "physical_health",\n "type": "string",\n "constraints": {\n "enum": [\n "Poor",\n "Fair",\n "Good",\n "Very good",\n "Excellent",\n ]\n }\n "enumOrdered": true\n }\n ],\n "missingValues": ["Don\'t know","Refused","Not applicable"]\n}\n')])])]),a("p",[e._v("This is our preferred strategy, as it provides all of the information"),a("br"),e._v("\nnecessary to support the categorical functionality of the software packages"),a("br"),e._v("\nlisted above, while still yielding a useable result for software without such"),a("br"),e._v("\ncapability. As described below, value labels or categoricals can be created"),a("br"),e._v("\nautomatically based on the ordering of the values in the "),a("code",[e._v("enum")]),e._v(" array, and the"),a("br"),e._v(" "),a("code",[e._v("missingValues")]),e._v(" can be incorporated into the value labels or categoricals if"),a("br"),e._v("\ndesired. In those cases where it is desired to have more control over how the"),a("br"),e._v("\nvalue labels are constructed, this information can be stored in a separate"),a("br"),e._v("\nfile in JSON format or as part of a custom extension to the table schema."),a("br"),e._v("\nSince such instructions do not describe the data themselves (but only how a"),a("br"),e._v("\nspecific software package should handle them), and since they are often"),a("br"),e._v("\nsoftware- and/or user-specific, we argue that they should not be included in"),a("br"),e._v("\nthe official table schema.")]),e._v(" "),a("p",[e._v("Alternatively, those who wish to store their data in encoded form (e.g., this"),a("br"),e._v("\nis the default for data exports from "),a("a",{attrs:{href:"https://projectredcap.org",target:"_blank",rel:"noopener noreferrer"}},[e._v("REDCap"),a("OutboundLink")],1),e._v(", a"),a("br"),e._v("\ncommonly-used platform for collecting data for clinical studies) may use"),a("br"),e._v("\nextension (2) to do so:")]),e._v(" "),a("div",{staticClass:"language- extra-class"},[a("pre",{pre:!0,attrs:{class:"language-text"}},[a("code",[e._v('{\n "fields": [\n {\n "name": "physical_health",\n "type": "integer",\n "constraints": {\n "enum": [1,2,3,4,5]\n }\n "enumOrdered": true,\n "enumLabels": {\n "1": "Poor",\n "2": "Fair",\n "3": "Good",\n "4": "Very good",\n "5": "Excellent"\n }\n }\n ],\n "missingValues": ["Don\'t know","Refused","Not applicable"]\n}\n')])])]),a("p",[e._v("Note that although the field type is "),a("code",[e._v("integer")]),e._v(", the keys in the "),a("code",[e._v("enumLabels")]),a("br"),e._v("\nobject must be wrapped in double quotes because this is required by the JSON"),a("br"),e._v("\nfile format.")]),e._v(" "),a("p",[e._v("A second variant of the example above is the following:")]),e._v(" "),a("div",{staticClass:"language- extra-class"},[a("pre",{pre:!0,attrs:{class:"language-text"}},[a("code",[e._v('{\n "fields": [\n {\n "name": "physical_health",\n "type": "integer",\n "constraints": {\n "enum": [1,2,3,4,5]\n }\n "enumOrdered": true,\n "enumLabels": {\n "1": "Poor",\n "2": "Fair",\n "3": "Good",\n "4": "Very good",\n "5": "Excellent",\n ".a": "Don\'t know",\n ".b": "Refused",\n ".c": "Not applicable"\n }\n }\n ],\n "missingValues": [".a",".b",".c"]\n}\n')])])]),a("p",[e._v("This represents encoded data exported from software with support for value"),a("br"),e._v("\nlabels. The values "),a("code",[e._v(".a")]),e._v(", "),a("code",[e._v(".b")]),e._v(", etc. are known as "),a("em",[e._v("extended missing values")]),a("br"),e._v("\n(Stata and SAS only) and provide 26 unique missing values for numeric fields"),a("br"),e._v('\n(both integer and float) in addition to the system missing value ("'),a("code",[e._v(".")]),e._v('"); in'),a("br"),e._v("\nSPSS these would be replaced with specially designated integers, typically"),a("br"),e._v("\nnegative (e.g., -97, -98 and -99).")]),e._v(" "),a("h3",{attrs:{id:"specification-15"}},[a("a",{staticClass:"header-anchor",attrs:{href:"#specification-15"}},[e._v("#")]),e._v(" Specification")]),e._v(" "),a("ol",[a("li",[a("p",[e._v("A field with an "),a("code",[e._v("enum")]),e._v(" constraint or an "),a("code",[e._v("enumLabels")]),e._v(" property MAY have an"),a("br"),e._v(" "),a("code",[e._v("enumOrdered")]),e._v(" property that MUST be a boolean. A value of "),a("code",[e._v("true")]),e._v(" indicates"),a("br"),e._v("\nthat the field should be treated as having an ordinal scale of measurement,"),a("br"),e._v("\nwith the ordering given by the order of the field’s "),a("code",[e._v("enum")]),e._v(" array or by the"),a("br"),e._v("\nlexical order of the "),a("code",[e._v("enumLabels")]),e._v(" object’s keys, with the latter taking"),a("br"),e._v("\nprecedence. Fields without an "),a("code",[e._v("enum")]),e._v(" constraint or an "),a("code",[e._v("enumLabels")]),e._v(" property"),a("br"),e._v("\nor for which the "),a("code",[e._v("enumLabels")]),e._v(" keys do not include all values observed"),a("br"),e._v("\nin the data (excluding any values specified in the "),a("code",[e._v("missingValues")]),a("br"),e._v("\nproperty) MUST NOT have an "),a("code",[e._v("enumOrdered")]),e._v(" property since in that case the"),a("br"),e._v("\ncorrect ordering of the data is ambiguous. The absence of an "),a("code",[e._v("enumOrdered")]),a("br"),e._v("\nproperty MUST NOT be taken to imply "),a("code",[e._v("enumOrdered: false")]),e._v(".")])]),e._v(" "),a("li",[a("p",[e._v("A field MAY have an "),a("code",[e._v("enumLabels")]),e._v(" property that MUST be an object. This"),a("br"),e._v("\nproperty SHOULD be used to indicate how the values in the data (represented"),a("br"),e._v("\nby the object’s keys) are to be labeled or translated (represented by the"),a("br"),e._v("\ncorresponding value). As required by the JSON format, the object’s keys"),a("br"),e._v("\nmust be listed as strings (i.e., wrapped in double quotes). The keys MAY"),a("br"),e._v("\ninclude values that do not appear in the data and MAY omit some values that"),a("br"),e._v("\ndo appear in the data. For clarity and to avoid unintentional loss of"),a("br"),e._v("\ninformation, the object’s values SHOULD be unique.")])])]),e._v(" "),a("h3",{attrs:{id:"suggested-implementations"}},[a("a",{staticClass:"header-anchor",attrs:{href:"#suggested-implementations"}},[e._v("#")]),e._v(" Suggested implementations")]),e._v(" "),a("p",[e._v("Note: The use cases below address only "),a("em",[e._v("reading data")]),e._v(" from a Frictionless data"),a("br"),e._v("\npackage; it is assumed that implementations will also provide the ability to"),a("br"),e._v("\nwrite Frictionless data packages using the schema extensions proposed above."),a("br"),e._v("\nWe suggest two types of implementations:")]),e._v(" "),a("ol",[a("li",[a("p",[e._v("Additions to the official Python Frictionless Framework to generate"),a("br"),e._v("\nsoftware-specific scripts that may be executed by a specific software"),a("br"),e._v("\npackage to read data from a Frictionless data package and create the"),a("br"),e._v("\nappropriate value labels or categoricals, as described below. These"),a("br"),e._v("\nscripts can then be included along with the data in the package itself.")])]),e._v(" "),a("li",[a("p",[e._v("Software-specific extension packages that may be installed to permit users"),a("br"),e._v("\nof that software to read data from a Frictionless data package directly,"),a("br"),e._v("\nautomatically creating the appropriate value labels or categoricals as"),a("br"),e._v("\ndescribed below.")])])]),e._v(" "),a("p",[e._v("The advantage of (1) is that it doesn’t require users to install another"),a("br"),e._v("\nsoftware package, which may in some cases be difficult or impossible. The"),a("br"),e._v("\nadvantage of (2) is that it provides native support for working with"),a("br"),e._v("\nFrictionless data packages, and may be both easier and faster once the package"),a("br"),e._v("\nis installed. We are in the process of implementing both approaches for Stata;"),a("br"),e._v("\nimplementations for the other software listed above are straightforward.")]),e._v(" "),a("h4",{attrs:{id:"software-that-supports-value-labels-stata-sas-or-spss"}},[a("a",{staticClass:"header-anchor",attrs:{href:"#software-that-supports-value-labels-stata-sas-or-spss"}},[e._v("#")]),e._v(" Software that supports value labels (Stata, SAS or SPSS)")]),e._v(" "),a("ol",[a("li",[a("p",[e._v("In cases where a field has an "),a("code",[e._v("enum")]),e._v(" constraint but no "),a("code",[e._v("enumLabels")]),a("br"),e._v("\nproperty, automatically generate a value label mapping the integers 1, 2,"),a("br"),e._v("\n3, … to the "),a("code",[e._v("enum")]),e._v(" values in order, use this to encode the field (thereby"),a("br"),e._v("\nchanging its type from "),a("code",[e._v("string")]),e._v(" to "),a("code",[e._v("integer")]),e._v("), and attach the value label"),a("br"),e._v("\nto the field. Provide option to skip automatically dropping values"),a("br"),e._v("\nspecified in the "),a("code",[e._v("missingValues")]),e._v(" property and instead add them in order to"),a("br"),e._v("\nthe end of the value label, encoded using extended missing values if"),a("br"),e._v("\nsupported.")])]),e._v(" "),a("li",[a("p",[e._v("In cases where the data are stored in encoded form (e.g., as integers) and"),a("br"),e._v("\na corresponding "),a("code",[e._v("enumLabels")]),e._v(" property is present, and assuming that the"),a("br"),e._v("\nkeys in the "),a("code",[e._v("enumLabels")]),e._v(" object are limited to integers and extended"),a("br"),e._v("\nmissing values (if supported), use the "),a("code",[e._v("enumLabels")]),e._v(" object to generate a"),a("br"),e._v("\nvalue label and attach it to the field. As with (1), provide option to skip"),a("br"),e._v("\nautomatically dropping values specified in the "),a("code",[e._v("missingValues")]),e._v(" property and"),a("br"),e._v("\ninstead add them in order to the end of the value label, encoded using"),a("br"),e._v("\nextended missing values if supported.")])]),e._v(" "),a("li",[a("p",[e._v("Although none of Stata, SAS or SPSS currently permits designating a specific"),a("br"),e._v("\nvariable as ordered, Stata permits attaching arbitrary metadata to"),a("br"),e._v("\nindividual variables. Thus, in cases where the "),a("code",[e._v("enumOrdered")]),e._v(" property is"),a("br"),e._v("\npresent, this information can be stored in Stata to inform the analyst and"),a("br"),e._v("\nto prevent loss of information when generating Frictionless data packages"),a("br"),e._v("\nfrom within Stata.")])])]),e._v(" "),a("h4",{attrs:{id:"software-that-supports-categoricals-or-factors-pandas-r-julia"}},[a("a",{staticClass:"header-anchor",attrs:{href:"#software-that-supports-categoricals-or-factors-pandas-r-julia"}},[e._v("#")]),e._v(" Software that supports categoricals or factors (Pandas, R, Julia)")]),e._v(" "),a("ol",[a("li",[a("p",[e._v("In cases where a field has an "),a("code",[e._v("enum")]),e._v(" constraint but no "),a("code",[e._v("enumLabels")]),a("br"),e._v("\nproperty, automatically define a categorical or factor using the "),a("code",[e._v("enum")]),a("br"),e._v("\nvalues in order, and convert the variable to categorical or factor type"),a("br"),e._v("\nusing this definition. Provide option to skip automatically dropping values"),a("br"),e._v("\nspecified in the "),a("code",[e._v("missingValues")]),e._v(" property and instead add them in order to"),a("br"),e._v("\nthe end of the "),a("code",[e._v("enum")]),e._v(" values when defining the categorical or factor.")])]),e._v(" "),a("li",[a("p",[e._v("In cases where the data are stored in encoded form (e.g., as integers) and"),a("br"),e._v("\na corresponding "),a("code",[e._v("enumLabels")]),e._v(" property is present, translate the data using"),a("br"),e._v("\nthe "),a("code",[e._v("enumLabels")]),e._v(" object, define a categorical or factor using the values of"),a("br"),e._v("\nthe "),a("code",[e._v("enumLabels")]),e._v(" object in lexical order of the keys, and convert the"),a("br"),e._v("\nvariable to categorical or factor type using this definition. Provide"),a("br"),e._v("\noption to skip automatically dropping values specified in the"),a("br"),e._v(" "),a("code",[e._v("missingValues")]),e._v(" property and instead add them to the end of the"),a("br"),e._v(" "),a("code",[e._v("enumLabels")]),e._v(" values when defining the categorical or factor.")])]),e._v(" "),a("li",[a("p",[e._v("In cases where a field has an "),a("code",[e._v("enumOrdered")]),e._v(" property, use that when"),a("br"),e._v("\ndefining the categorical or factor.")])])]),e._v(" "),a("h4",{attrs:{id:"all-software"}},[a("a",{staticClass:"header-anchor",attrs:{href:"#all-software"}},[e._v("#")]),e._v(" All software")]),e._v(" "),a("p",[e._v("Although the extensions proposed here are intended primarily to support the"),a("br"),e._v("\nuse of value labels and categoricals in software that supports them, they also"),a("br"),e._v("\nprovide additional functionality when reading data into any software that can"),a("br"),e._v("\nhandle tabular data. Specifically, the "),a("code",[e._v("enumLabels")]),e._v(" property may be used to"),a("br"),e._v("\nsupport the use of enums even in cases where value labels or categoricals are"),a("br"),e._v("\nnot being used. For example, it is standard practice in software for analyzing"),a("br"),e._v("\ngenetic data to code sex as 0, 1 and 2 (corresponding to “Unknown”, “Male” and"),a("br"),e._v("\n“Female”) and affection status as 0, 1 and 2 (corresponding to “Unknown”,"),a("br"),e._v("\n“Unaffected” and “Affected”). In such cases, the "),a("code",[e._v("enumLabels")]),e._v(" property may be"),a("br"),e._v("\nused to confirm that the data follow the standard convention or to indicate"),a("br"),e._v("\nthat they deviate from it; it may also be used to translate those codes into"),a("br"),e._v("\nhuman-readable values, if desired.")]),e._v(" "),a("h3",{attrs:{id:"notes"}},[a("a",{staticClass:"header-anchor",attrs:{href:"#notes"}},[e._v("#")]),e._v(" Notes")]),e._v(" "),a("p",[e._v("While this pattern is designed as an extension to "),a("a",{attrs:{href:"https://specs.frictionlessdata.io/table-schema/",target:"_blank",rel:"noopener noreferrer"}},[e._v("Table Schema"),a("OutboundLink")],1),e._v(" fields, it could also be used to document "),a("code",[e._v("enum")]),e._v(" values of properties in "),a("a",{attrs:{href:"https://specs.frictionlessdata.io/profiles/",target:"_blank",rel:"noopener noreferrer"}},[e._v("profiles"),a("OutboundLink")],1),e._v(", such as contributor roles.")]),e._v(" "),a("p",[e._v("This pattern originally included a proposal to add an optional field-specific"),a("br"),e._v(" "),a("code",[e._v("missingValues")]),e._v(" property similar to that described in the pattern"),a("br"),e._v("\n“"),a("a",{attrs:{href:"https://specs.frictionlessdata.io/patterns/#missing-values-per-field",target:"_blank",rel:"noopener noreferrer"}},[e._v("missing values per field"),a("OutboundLink")],1),e._v("”"),a("br"),e._v("\nappearing in this document above. The objective was to provide a mechnanism to"),a("br"),e._v("\ndistinguish between so-called "),a("em",[e._v("system missing values")]),e._v(" (i.e., values that"),a("br"),e._v("\nindicate only that the corresponding data are missing) and other values that"),a("br"),e._v("\nconvey meaning but are typically excluded when fitting statistical models. The"),a("br"),e._v("\nlatter may be represented by "),a("em",[e._v("extended missing values")]),e._v(" ("),a("code",[e._v(".a")]),e._v(", "),a("code",[e._v(".b")]),e._v(", "),a("code",[e._v(".c")]),e._v(","),a("br"),e._v("\netc.) in Stata and SAS, or in SPSS by negative integers that are then"),a("br"),e._v("\ndesignated as missing by using the "),a("code",[e._v("MISSING VALUES")]),e._v(" command. For example,"),a("br"),e._v("\nvalues such as “NA”, “Not applicable”, “.”, etc. could be specified in the"),a("br"),e._v("\nresource level "),a("code",[e._v("missingValues")]),e._v(" property, while values such as “Don’t know” and"),a("br"),e._v("\n“Refused”—often used when generating tabular summaries and occasionally used"),a("br"),e._v("\nwhen fitting certain statistical models—could be specified in the"),a("br"),e._v("\ncorresponding field level "),a("code",[e._v("missingValues")]),e._v(" property. The former would still be"),a("br"),e._v("\nconverted to "),a("code",[e._v("null")]),e._v(" before type-specific string conversion (just as they are"),a("br"),e._v("\nnow), while the latter could be used by capable software when creating value"),a("br"),e._v("\nlabels or categoricals.")]),e._v(" "),a("p",[e._v("While this proposal was consistent with the principles outlined at the"),a("br"),e._v("\nbeginning (in particular, existing software would still yield a usable result"),a("br"),e._v("\nwhen reading the data), we realized that it would conflict with what appears"),a("br"),e._v("\nto be an emerging consensus regarding field-specific "),a("code",[e._v("missingValues")]),e._v("; i.e.,"),a("br"),e._v("\nthat they should "),a("em",[e._v("replace")]),e._v(" the less specific resource level "),a("code",[e._v("missingValues")]),a("br"),e._v("\nfor the corresponding field rather than be combined with them (see the discussion"),a("br"),e._v(" "),a("a",{attrs:{href:"https://github.com/frictionlessdata/specs/issues/551",target:"_blank",rel:"noopener noreferrer"}},[e._v("here"),a("OutboundLink")],1),e._v(" as well as the"),a("br"),e._v("\n“"),a("a",{attrs:{href:"https://specs.frictionlessdata.io/patterns/#missing-values-per-field",target:"_blank",rel:"noopener noreferrer"}},[e._v("missing values per field"),a("OutboundLink")],1),e._v("”"),a("br"),e._v("\npattern above). While there is good reason for replacing rather than combining"),a("br"),e._v("\nhere (e.g., it is more explicit), it would unfortunately conflict with the"),a("br"),e._v("\nidea of using the field-specific "),a("code",[e._v("missingValues")]),e._v(" in conjunction with the"),a("br"),e._v("\nresource level "),a("code",[e._v("missingValues")]),e._v(" as just described; namely, if the"),a("br"),e._v("\nfield-specific property replaced the resource level property then the system"),a("br"),e._v("\nmissing values would no longer be converted to "),a("code",[e._v("null")]),e._v(", as desired.")]),e._v(" "),a("p",[e._v("For this reason, we have dropped the proposal to add a field-specific"),a("br"),e._v(" "),a("code",[e._v("missingValues")]),e._v(" property from this pattern, and assert that implementation of"),a("br"),e._v("\nthis pattern by software should assume that if a field-specific "),a("code",[e._v("missingValues")]),a("br"),e._v("\nproperty is added to the"),a("br"),e._v(" "),a("a",{attrs:{href:"https://specs.frictionlessdata.io//table-schema/",target:"_blank",rel:"noopener noreferrer"}},[e._v("table schema"),a("OutboundLink")],1),a("br"),e._v("\nit should, if present, replace the resource level "),a("code",[e._v("missingValues")]),e._v(" property for"),a("br"),e._v("\nthe corresponding field. We do not believe that this change represents a"),a("br"),e._v("\nsubstantial limitation when creating value labels or categoricals, since"),a("br"),e._v("\nsystem missing values can typically be easily distinguished from other missing"),a("br"),e._v("\nvalues when exported in CSV format (e.g., “.” in Stata or SAS, “NA” in R, or"),a("br"),e._v("\n“” in Pandas).")])])}),[],!1,null,null,null);t.default=n.exports}}]); \ No newline at end of file +(window.webpackJsonp=window.webpackJsonp||[]).push([[24],{394:function(e,t,a){"use strict";a.r(t);var s=a(26),n=Object(s.a)({},(function(){var e=this,t=e.$createElement,a=e._self._c||t;return a("ContentSlotsDistributor",{attrs:{"slot-key":e.$parent.slotKey}},[a("h1",{attrs:{id:"patterns"}},[a("a",{staticClass:"header-anchor",attrs:{href:"#patterns"}},[e._v("#")]),e._v(" Patterns")]),e._v(" "),a("p",[e._v("This document describes various patterns for solving common problems, in ways that are not (yet) specified in any Frictionless Data specification. If we see increased adoption, or wide support, for any pattern, it is a prime candidate for formalising as part of a specification.")]),e._v(" "),a("h2",{attrs:{id:"private-properties"}},[a("a",{staticClass:"header-anchor",attrs:{href:"#private-properties"}},[e._v("#")]),e._v(" Private properties")]),e._v(" "),a("h3",{attrs:{id:"overview"}},[a("a",{staticClass:"header-anchor",attrs:{href:"#overview"}},[e._v("#")]),e._v(" Overview")]),e._v(" "),a("p",[e._v("Some software that implements the Frictionless Data specifications may need to store additional information on the various Frictionless Data descriptors.")]),e._v(" "),a("p",[e._v("For example, a data registry that provides metadata via "),a("code",[e._v("datapackage.json")]),e._v(" may wish to set an internal version or identifier that is system-specific, and should not be considered as part of the user-generated metadata.")]),e._v(" "),a("p",[e._v("Properties to store such information should be considered “private”, and by convention, the names should be prefixed by an underscore "),a("code",[e._v("_")]),e._v(".")]),e._v(" "),a("h3",{attrs:{id:"implementations"}},[a("a",{staticClass:"header-anchor",attrs:{href:"#implementations"}},[e._v("#")]),e._v(" Implementations")]),e._v(" "),a("p",[e._v("There are no known implementations at present.")]),e._v(" "),a("h3",{attrs:{id:"specification"}},[a("a",{staticClass:"header-anchor",attrs:{href:"#specification"}},[e._v("#")]),e._v(" Specification")]),e._v(" "),a("p",[e._v("On any Frictionless Data descriptor, data that is not generated by the author/contributors, but is generated by software/a system handling the data, "),a("code",[e._v("SHOULD")]),e._v(" be considered as “private”, and be prefixed by an underscore "),a("code",[e._v("_")]),e._v(".")]),e._v(" "),a("p",[e._v("To demonstrate, let’s take the example of a data registry that implements "),a("code",[e._v("datapackage.json")]),e._v(" for storing dataset metadata.")]),e._v(" "),a("p",[e._v("A user might upload a "),a("code",[e._v("datapackage.json")]),e._v(" as follows:")]),e._v(" "),a("div",{staticClass:"language- extra-class"},[a("pre",{pre:!0,attrs:{class:"language-text"}},[a("code",[e._v('{\n "name": "my-package",\n "resources": [\n {\n "name": "my-resource",\n "data": [ "my-resource.csv" ]\n }\n ]\n}\n')])])]),a("p",[e._v("The registry itself may have a platform-specific version system, and increment versions on each update of the data. To store this information on the datapackage itself, the platform could save this information in a “private” "),a("code",[e._v("_platformVersion")]),e._v(" property as follows:")]),e._v(" "),a("div",{staticClass:"language- extra-class"},[a("pre",{pre:!0,attrs:{class:"language-text"}},[a("code",[e._v('{\n "name": "my-package",\n "_platformVersion": 7\n "resources": [\n {\n "name": "my-resource",\n "data": [ "my-resource.csv" ]\n }\n ]\n}\n')])])]),a("p",[e._v("Usage of “private” properties ensures a clear distinction between data stored on the descriptor that is user (author/contributor) defined, and any additional data that may be stored by a 3rd party.")]),e._v(" "),a("h2",{attrs:{id:"caching-of-resources"}},[a("a",{staticClass:"header-anchor",attrs:{href:"#caching-of-resources"}},[e._v("#")]),e._v(" Caching of resources")]),e._v(" "),a("h3",{attrs:{id:"overview-2"}},[a("a",{staticClass:"header-anchor",attrs:{href:"#overview-2"}},[e._v("#")]),e._v(" Overview")]),e._v(" "),a("p",[e._v("All Frictionless Data specifications allow for referencing resources via http or a local filesystem.")]),e._v(" "),a("p",[e._v("In the case of remote resources via http, there is always the possibility that the remote server will be unavailable, or, that the resource itself will be temporarily or permanently removed.")]),e._v(" "),a("p",[e._v("Applications that are concerned with the persistent storage of data described in Frictionless Data specifications can use a "),a("code",[e._v("_cache")]),e._v(" property that mirrors the functionality and usage of the "),a("code",[e._v("data")]),e._v(" property, and refers to a storage location for the data that the application can fall back to if the canonical resource is unavailable.")]),e._v(" "),a("h3",{attrs:{id:"implementations-2"}},[a("a",{staticClass:"header-anchor",attrs:{href:"#implementations-2"}},[e._v("#")]),e._v(" Implementations")]),e._v(" "),a("p",[e._v("There are no known implementations of this pattern at present.")]),e._v(" "),a("h3",{attrs:{id:"specification-2"}},[a("a",{staticClass:"header-anchor",attrs:{href:"#specification-2"}},[e._v("#")]),e._v(" Specification")]),e._v(" "),a("p",[e._v("Implementations "),a("code",[e._v("MAY")]),e._v(" handle a "),a("code",[e._v("_cache")]),e._v(" property on any descriptor that supports a "),a("code",[e._v("data")]),e._v(" property. In the case that the data referenced in "),a("code",[e._v("data")]),e._v(" is unavailable, "),a("code",[e._v("_cache")]),e._v(" should be used as a fallback to access the data. The handling of the data stored at "),a("code",[e._v("_cache")]),e._v(" is beyond the scope of the specification. Implementations might store a copy of the resources in "),a("code",[e._v("data")]),e._v(" at ingestion time, update at regular intervals, or any other method to keep an up-to-date, persistent copy.")]),e._v(" "),a("p",[e._v("Some examples of the "),a("code",[e._v("_cache")]),e._v(" property.")]),e._v(" "),a("div",{staticClass:"language- extra-class"},[a("pre",{pre:!0,attrs:{class:"language-text"}},[a("code",[e._v('{\n "name": "my-package",\n "resources": [\n {\n "name": "my-resource",\n "data": [ "http://example.com/data/csv/my-resource.csv" ],\n "_cache": "my-resource.csv"\n },\n {\n "name": "my-resource",\n "data": [ "http://example.com/data/csv/my-resource.csv" ],\n "_cache": "http://data.registry.com/user/files/my-resource.csv"\n },\n {\n "name": "my-resource",\n "data": [\n "http://example.com/data/csv/my-resource.csv",\n "http://somewhere-else.com/data/csv/resource2.csv"\n ],\n "_cache": [\n "my-resource.csv",\n "resource2.csv"\n ]\n },\n {\n "name": "my-resource",\n "data": [ "http://example.com/data/csv/my-resource.csv" ],\n "_cache": "my-resource.csv"\n }\n ]\n}\n')])])]),a("h2",{attrs:{id:"compression-of-resources"}},[a("a",{staticClass:"header-anchor",attrs:{href:"#compression-of-resources"}},[e._v("#")]),e._v(" Compression of resources")]),e._v(" "),a("h3",{attrs:{id:"overview-3"}},[a("a",{staticClass:"header-anchor",attrs:{href:"#overview-3"}},[e._v("#")]),e._v(" Overview")]),e._v(" "),a("p",[e._v("It can be argued that applying compression to data resources can make data package publishing more cost-effective and sustainable. Compressing data resources gives publishers the benefit of reduced storage and bandwidth costs and gives consumers the benefit of shorter download times.")]),e._v(" "),a("h3",{attrs:{id:"implementations-3"}},[a("a",{staticClass:"header-anchor",attrs:{href:"#implementations-3"}},[e._v("#")]),e._v(" Implementations")]),e._v(" "),a("ul",[a("li",[a("a",{attrs:{href:"https://github.com/frictionlessdata/tabulator-py",target:"_blank",rel:"noopener noreferrer"}},[e._v("tabulator-py (Gzip and Zip support)"),a("OutboundLink")],1)]),e._v(" "),a("li",[a("a",{attrs:{href:"https://github.com/nimblelearn/datapackage-connector",target:"_blank",rel:"noopener noreferrer"}},[e._v("datapackage-connector (Gzip support)"),a("OutboundLink")],1)]),e._v(" "),a("li",[a("a",{attrs:{href:"https://github.com/nimblelearn/datapackage-m",target:"_blank",rel:"noopener noreferrer"}},[e._v("datapackage-m (Gzip support)"),a("OutboundLink")],1)])]),e._v(" "),a("h3",{attrs:{id:"specification-3"}},[a("a",{staticClass:"header-anchor",attrs:{href:"#specification-3"}},[e._v("#")]),e._v(" Specification")]),e._v(" "),a("p",[e._v("All compressed resources "),a("code",[e._v("MUST")]),e._v(" have a "),a("code",[e._v("path")]),e._v(" that allows the "),a("code",[e._v("compression")]),e._v(" property to be inferred. If the compression can’t be inferred from the "),a("code",[e._v("path")]),e._v(" property (e.g. a custom file extension is used) then the "),a("code",[e._v("compression")]),e._v(" property "),a("code",[e._v("MUST")]),e._v(" be used to specify the compression.")]),e._v(" "),a("p",[e._v("Supported compression types:")]),e._v(" "),a("ul",[a("li",[e._v("gz")]),e._v(" "),a("li",[e._v("zip")])]),e._v(" "),a("p",[e._v("Example of a compressed resource with implied compression:")]),e._v(" "),a("div",{staticClass:"language- extra-class"},[a("pre",{pre:!0,attrs:{class:"language-text"}},[a("code",[e._v('{\n "name": "data-resource-compression-example",\n "path": "http://example.com/large-data-file.csv.gz",\n "title": "Large Data File",\n "description": "This large data file benefits from compression.",\n "format": "csv",\n "mediatype": "text/csv",\n "encoding": "utf-8",\n "bytes": 1073741824\n}\n')])])]),a("p",[e._v("Example of a compressed resource with the "),a("code",[e._v("compression")]),e._v(" property:")]),e._v(" "),a("div",{staticClass:"language- extra-class"},[a("pre",{pre:!0,attrs:{class:"language-text"}},[a("code",[e._v('{\n "name": "data-resource-compression-example",\n "path": "http://example.com/large-data-file.csv.gz",\n "title": "Large Data File",\n "description": "This large data file benefits from compression.",\n "format": "csv",\n "compression" : "gz",\n "mediatype": "text/csv",\n "encoding": "utf-8",\n "bytes": 1073741824\n}\n')])])]),a("div",{staticClass:"custom-block tip"},[a("p",{staticClass:"custom-block-title"},[e._v("NOTE")]),e._v(" "),a("p",[e._v("Resource properties e.g. bytes, hash etc apply to the compressed object – not to the original uncompressed object.")])]),e._v(" "),a("h2",{attrs:{id:"language-support"}},[a("a",{staticClass:"header-anchor",attrs:{href:"#language-support"}},[e._v("#")]),e._v(" Language support")]),e._v(" "),a("h3",{attrs:{id:"overview-4"}},[a("a",{staticClass:"header-anchor",attrs:{href:"#overview-4"}},[e._v("#")]),e._v(" Overview")]),e._v(" "),a("p",[e._v("Language support is a different concern to translation support. Language support deals with declaring the default language of a descriptor and the data it contains in the resources array. Language support makes no claim about the presence of translations when one or more languages are supported in a descriptor or in data. Via the introduction of a "),a("code",[e._v("languages")]),e._v(" array to any descriptor, we can declare the default language, and any other languages that "),a("code",[e._v("SHOULD")]),e._v(" be found in the descriptor and the data.")]),e._v(" "),a("h3",{attrs:{id:"implementations-4"}},[a("a",{staticClass:"header-anchor",attrs:{href:"#implementations-4"}},[e._v("#")]),e._v(" Implementations")]),e._v(" "),a("p",[e._v("There are no known implementations of this pattern at present.")]),e._v(" "),a("h3",{attrs:{id:"specification-4"}},[a("a",{staticClass:"header-anchor",attrs:{href:"#specification-4"}},[e._v("#")]),e._v(" Specification")]),e._v(" "),a("p",[e._v("Any Frictionless Data descriptor can declare the language configuration of its metadata and data with the "),a("code",[e._v("languages")]),e._v(" array.")]),e._v(" "),a("p",[a("code",[e._v("languages")]),e._v(" "),a("code",[e._v("MUST")]),e._v(" be an array, and the first item in the array is the default (non-translated) language.")]),e._v(" "),a("p",[e._v("If no "),a("code",[e._v("languages")]),e._v(" array is present, the default language is English ("),a("code",[e._v("en")]),e._v("), and therefore is equivalent to:")]),e._v(" "),a("div",{staticClass:"language- extra-class"},[a("pre",{pre:!0,attrs:{class:"language-text"}},[a("code",[e._v('{\n "name": "my-package",\n "languages": ["en"]\n}\n')])])]),a("p",[e._v("The presence of a languages array does not ensure that the metadata or the data has translations for all supported languages.")]),e._v(" "),a("p",[e._v("The descriptor and data sources "),a("code",[e._v("MUST")]),e._v(" be in the default language. The descriptor and data sources "),a("code",[e._v("MAY")]),e._v(" have translations for the other languages in the array, using the same language code. "),a("code",[e._v("IF")]),e._v(" a translation is not present, implementing code "),a("code",[e._v("MUST")]),e._v(" fallback to the default language string.")]),e._v(" "),a("p",[e._v("Example usage of "),a("code",[e._v("languages")]),e._v(", implemented in the metadata of a descriptor:")]),e._v(" "),a("div",{staticClass:"language- extra-class"},[a("pre",{pre:!0,attrs:{class:"language-text"}},[a("code",[e._v('{\n "name": "sun-package",\n "languages": ["es", "en"],\n "title": "Sol"\n}\n\n# which is equivalent to\n{\n "name": "sun-package",\n "languages": ["es", "en"],\n "title": {\n "": "Sol",\n "en": "Sun"\n }\n}\n')])])]),a("p",[e._v("Example usage of "),a("code",[e._v("languages")]),e._v(" implemented in the data described by a resource:")]),e._v(" "),a("div",{staticClass:"language- extra-class"},[a("pre",{pre:!0,attrs:{class:"language-text"}},[a("code",[e._v('# resource descriptor\n{\n "name": "solar-system",\n "data": [ "solar-system.csv" ]\n "fields": [\n ...\n ],\n "languages": ["es", "en", "he", "fr", "ar"]\n}\n\n# data source\n# some languages have translations, some do not\n# assumes a certain translation pattern, see the related section\nid,name,name@fr,name@he,name@en\n1,Sol,Soleil,שמש,Sun\n2,Luna,Lune,ירח,Moon\n')])])]),a("h2",{attrs:{id:"translation-support"}},[a("a",{staticClass:"header-anchor",attrs:{href:"#translation-support"}},[e._v("#")]),e._v(" Translation support")]),e._v(" "),a("h3",{attrs:{id:"overview-5"}},[a("a",{staticClass:"header-anchor",attrs:{href:"#overview-5"}},[e._v("#")]),e._v(" Overview")]),e._v(" "),a("p",[e._v("Following on from a general pattern for language support, and the explicit support of metadata translations in Frictionless Data descriptors, it would be desirable to support translations in source data.")]),e._v(" "),a("p",[e._v("We currently have two patterns for this in discussion. Both patterns arise from real-world implementations that are not specifically tied to Frictionless Data.")]),e._v(" "),a("p",[e._v("One pattern suggests inline translations with the source data, reserving the "),a("code",[e._v("@")]),e._v(" symbol in the naming of fields to denote translations.")]),e._v(" "),a("p",[e._v("The other describes a pattern for storing additional translation sources, co-located with the “source” file described in a descriptor "),a("code",[e._v("data")]),e._v(".")]),e._v(" "),a("h3",{attrs:{id:"implementations-5"}},[a("a",{staticClass:"header-anchor",attrs:{href:"#implementations-5"}},[e._v("#")]),e._v(" Implementations")]),e._v(" "),a("p",[e._v("There are no known implementations of this pattern in the Frictionless Data core libraries at present.")]),e._v(" "),a("h3",{attrs:{id:"specification-5"}},[a("a",{staticClass:"header-anchor",attrs:{href:"#specification-5"}},[e._v("#")]),e._v(" Specification")]),e._v(" "),a("h4",{attrs:{id:"inline"}},[a("a",{staticClass:"header-anchor",attrs:{href:"#inline"}},[e._v("#")]),e._v(" Inline")]),e._v(" "),a("p",[a("strong",[e._v("Uses a column naming convention for accessing translations")]),e._v(".")]),e._v(" "),a("p",[e._v("Tabular resource descriptors support translations using "),a("code",[e._v("{field_name}@{lang_code}")]),e._v(" syntax for translated field names. "),a("code",[e._v("lang_code")]),e._v(" "),a("code",[e._v("MUST")]),e._v(" be present in the "),a("code",[e._v("languages")]),e._v(" array that applies to the resource.")]),e._v(" "),a("p",[e._v("Any field with the "),a("code",[e._v("@")]),e._v(" symbol "),a("code",[e._v("MUST")]),e._v(" be a translation field for another field of data, and "),a("code",[e._v("MUST")]),e._v(" be parsable according to the "),a("code",[e._v("{field_name}@{lang_code}")]),e._v(" pattern.")]),e._v(" "),a("p",[e._v("If a translation field is found in the data that does not have a corresponding "),a("code",[e._v("field")]),e._v(" (e.g.: "),a("code",[e._v("title@es")]),e._v(" but no "),a("code",[e._v("title")]),e._v("), then the translation field "),a("code",[e._v("SHOULD")]),e._v(" be ignored.")]),e._v(" "),a("p",[e._v("If a translation field is found in the data that uses a "),a("code",[e._v("lang_code")]),e._v(" "),a("em",[e._v("not")]),e._v(" declared in the applied "),a("code",[e._v("languages")]),e._v(" array, then the translation field "),a("code",[e._v("SHOULD")]),e._v(" be ignored.")]),e._v(" "),a("p",[e._v("Translation fields "),a("code",[e._v("MUST NOT")]),e._v(" be described in a schema "),a("code",[e._v("fields")]),e._v(" array.")]),e._v(" "),a("p",[e._v("Translation fields "),a("code",[e._v("MUST")]),e._v(" match the "),a("code",[e._v("type")]),e._v(", "),a("code",[e._v("format")]),e._v(" and "),a("code",[e._v("constraints")]),e._v(" of the field they translate, with a single exception: Translation fields are never required, and therefore "),a("code",[e._v("constraints.required")]),e._v(" is always "),a("code",[e._v("false")]),e._v(" for a translation field.")]),e._v(" "),a("h4",{attrs:{id:"co-located-translation-sources"}},[a("a",{staticClass:"header-anchor",attrs:{href:"#co-located-translation-sources"}},[e._v("#")]),e._v(" Co-located translation sources")]),e._v(" "),a("p",[a("strong",[e._v("Uses a file storage convention for accessing translations")]),e._v(".")]),e._v(" "),a("p",[e._v("To be contributed by @jheeffer")]),e._v(" "),a("ul",[a("li",[e._v("Has to handle local and remote resources")]),e._v(" "),a("li",[e._v("Has to be explicit about the translation key/value pattern in the translation files")])]),e._v(" "),a("div",{staticClass:"language- extra-class"},[a("pre",{pre:!0,attrs:{class:"language-text"}},[a("code",[e._v("# local\ndata/file1.csv\ndata/lang/file1-en.csv\ndata/lang/file1-es.csv\n\n# remote\nhttp://example/com/data/file2.csv\nhttp://example/com/data/lang/file2-en.csv\nhttp://example/com/data/lang/file2-es.csv\n")])])]),a("h2",{attrs:{id:"table-schema-foreign-keys-to-data-packages"}},[a("a",{staticClass:"header-anchor",attrs:{href:"#table-schema-foreign-keys-to-data-packages"}},[e._v("#")]),e._v(" Table Schema: Foreign Keys to Data Packages")]),e._v(" "),a("h3",{attrs:{id:"overview-6"}},[a("a",{staticClass:"header-anchor",attrs:{href:"#overview-6"}},[e._v("#")]),e._v(" Overview")]),e._v(" "),a("p",[e._v("A foreign key is a reference where values in a field (or fields) in a Tabular Data Resource link to values in a field (or fields) in a Tabular Data Resource in the same or in another Tabular Data Package.")]),e._v(" "),a("p",[e._v("This pattern allows users to link values in a field (or fields) in a Tabular Data Resource to values in a field (or fields) in a Tabular Data Resource in a different Tabular Data Package.")]),e._v(" "),a("h3",{attrs:{id:"specification-6"}},[a("a",{staticClass:"header-anchor",attrs:{href:"#specification-6"}},[e._v("#")]),e._v(" Specification")]),e._v(" "),a("p",[e._v("The "),a("RouterLink",{attrs:{to:"/table-schema/#foreign-keys"}},[a("code",[e._v("foreignKeys")])]),e._v(" array MAY have a property "),a("code",[e._v("package")]),e._v(". This property MUST be, either:")],1),e._v(" "),a("ul",[a("li",[e._v("a string that is a fully qualified HTTP address to a Data Package "),a("code",[e._v("datapackage.json")]),e._v(" file")]),e._v(" "),a("li",[e._v("a data package "),a("RouterLink",{attrs:{to:"/data-package/#name"}},[a("code",[e._v("name")])]),e._v(" that can be resolved by a canonical data package registry")],1)]),e._v(" "),a("p",[e._v("If the referenced data package has an "),a("RouterLink",{attrs:{to:"/data-package/#id"}},[a("code",[e._v("id")])]),e._v(" that is a fully qualified HTTP address, it SHOULD be used as the "),a("code",[e._v("package")]),e._v(" value.")],1),e._v(" "),a("p",[e._v("For example:")]),e._v(" "),a("div",{staticClass:"language- extra-class"},[a("pre",{pre:!0,attrs:{class:"language-text"}},[a("code",[e._v('"foreignKeys": [{\n "fields": ["code"],\n "reference": {\n "package": "https://raw.githubusercontent.com/frictionlessdata/example-data-packages/master/donation-codes/datapackage.json",\n "resource": "donation-codes",\n "fields": ["donation code"]\n }\n }]\n')])])]),a("h2",{attrs:{id:"data-package-version"}},[a("a",{staticClass:"header-anchor",attrs:{href:"#data-package-version"}},[e._v("#")]),e._v(" Data Package Version")]),e._v(" "),a("h3",{attrs:{id:"specification-7"}},[a("a",{staticClass:"header-anchor",attrs:{href:"#specification-7"}},[e._v("#")]),e._v(" Specification")]),e._v(" "),a("p",[e._v("The Data Package version format follows the "),a("a",{attrs:{href:"http://semver.org",target:"_blank",rel:"noopener noreferrer"}},[e._v("Semantic Versioning"),a("OutboundLink")],1),e._v(" specification format: MAJOR.MINOR.PATCH")]),e._v(" "),a("p",[e._v("The version numbers, and the way they change, convey meaning about how the data package has been modified from one version to the next.")]),e._v(" "),a("p",[e._v("Given a Data Package version number MAJOR.MINOR.PATCH, increment the:")]),e._v(" "),a("p",[e._v("MAJOR version when you make incompatible changes, e.g.")]),e._v(" "),a("ul",[a("li",[e._v("Change the data package, resource or field "),a("code",[e._v("name")]),e._v(" or "),a("code",[e._v("identifier")])]),e._v(" "),a("li",[e._v("Add, remove or re-order fields")]),e._v(" "),a("li",[e._v("Change a field "),a("code",[e._v("type")]),e._v(" or "),a("code",[e._v("format")])]),e._v(" "),a("li",[e._v("Change a field "),a("code",[e._v("constraint")]),e._v(" to be more restrictive")]),e._v(" "),a("li",[e._v("Combine, split, delete or change the meaning of data that is referenced by another data resource")])]),e._v(" "),a("p",[e._v("MINOR version when you add data or change metadata in a backwards-compatible manner, e.g.")]),e._v(" "),a("ul",[a("li",[e._v("Add a new data resource to a data package")]),e._v(" "),a("li",[e._v("Add new data to an existing data resource")]),e._v(" "),a("li",[e._v("Change a field "),a("code",[e._v("constraint")]),e._v(" to be less restrictive")]),e._v(" "),a("li",[e._v("Update a reference to another data resource")]),e._v(" "),a("li",[e._v("Change data to reflect changes in referenced data")])]),e._v(" "),a("p",[e._v("PATCH version when you make backwards-compatible fixes, e.g.")]),e._v(" "),a("ul",[a("li",[e._v("Correct errors in existing data")]),e._v(" "),a("li",[e._v("Change descriptive metadata properties")])]),e._v(" "),a("h3",{attrs:{id:"scenarios"}},[a("a",{staticClass:"header-anchor",attrs:{href:"#scenarios"}},[e._v("#")]),e._v(" Scenarios")]),e._v(" "),a("ul",[a("li",[e._v("You are developing your data though public consultation. Start your initial data release at 0.1.0")]),e._v(" "),a("li",[e._v("You release your data for the first time. Use version 1.0.0")]),e._v(" "),a("li",[e._v("You append last months data to an existing release. Increment the MINOR version number")]),e._v(" "),a("li",[e._v("You append a column to the data. Increment the MAJOR version number")]),e._v(" "),a("li",[e._v("You relocate the data to a new "),a("code",[e._v("URL")]),e._v(" or "),a("code",[e._v("path")]),e._v(". No change in the version number")]),e._v(" "),a("li",[e._v("You change a "),a("code",[e._v("title")]),e._v(", "),a("code",[e._v("description")]),e._v(", or other descriptive metadata. Increment the PATCH version")]),e._v(" "),a("li",[e._v("You fix a data entry error by modifying a value. Increment the PATCH version")]),e._v(" "),a("li",[e._v("You split a row of data in a foreign key reference table. Increment the MAJOR version number")]),e._v(" "),a("li",[e._v("You update the data and schema to refer to a new version of a foreign key reference table. Increment the MINOR version number")])]),e._v(" "),a("h2",{attrs:{id:"data-dependencies"}},[a("a",{staticClass:"header-anchor",attrs:{href:"#data-dependencies"}},[e._v("#")]),e._v(" Data Dependencies")]),e._v(" "),a("p",[e._v("Consider a situation where data packages are part of a tool chain that, say, loads all of the data into an SQL db. You can then imagine a situation where one requires package A which requires package B + C.")]),e._v(" "),a("p",[e._v("In this case you want to specify that A depends on B and C – and that “installing” A should install B and C. This is the purpose of "),a("code",[e._v("dataDependencies")]),e._v(" property.")]),e._v(" "),a("h3",{attrs:{id:"specification-8"}},[a("a",{staticClass:"header-anchor",attrs:{href:"#specification-8"}},[e._v("#")]),e._v(" Specification")]),e._v(" "),a("p",[a("code",[e._v("dataDependencies")]),e._v(" is an object. It follows same format as CommonJS Packages spec v1.1. Each dependency defines the lowest compatible MAJOR[.MINOR[.PATCH]] dependency versions (only one per MAJOR version) with which the package has been tested and is assured to work. The version may be a simple version string (see the version property for acceptable forms), or it may be an object group of dependencies which define a set of options, any one of which satisfies the dependency. The ordering of the group is significant and earlier entries have higher priority. Example:")]),e._v(" "),a("div",{staticClass:"language-javascript extra-class"},[a("pre",{pre:!0,attrs:{class:"language-javascript"}},[a("code",[a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"dataDependencies"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("{")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"country-codes"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('""')]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v(",")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"unemployment"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"2.1"')]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v(",")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"geo-boundaries"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("{")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"acmecorp-geo-boundaries"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("[")]),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"1.0"')]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v(",")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"2.0"')]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("]")]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v(",")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"othercorp-geo-boundaries"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"0.9.8"')]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v(",")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("}")]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v(",")]),e._v("\n"),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("}")]),e._v("\n")])])]),a("h3",{attrs:{id:"implementations-6"}},[a("a",{staticClass:"header-anchor",attrs:{href:"#implementations-6"}},[e._v("#")]),e._v(" Implementations")]),e._v(" "),a("p",[e._v("None known.")]),e._v(" "),a("h2",{attrs:{id:"table-schema-metadata-properties"}},[a("a",{staticClass:"header-anchor",attrs:{href:"#table-schema-metadata-properties"}},[e._v("#")]),e._v(" Table Schema: metadata properties")]),e._v(" "),a("h3",{attrs:{id:"overview-7"}},[a("a",{staticClass:"header-anchor",attrs:{href:"#overview-7"}},[e._v("#")]),e._v(" Overview")]),e._v(" "),a("p",[e._v("Table Schemas need their own metadata to be stand-alone and interpreted without relying on other contextual information (Data Package metadata for example). Adding metadata to describe schemas in a structured way would help users to understand them and would increase their sharing and reuse.")]),e._v(" "),a("p",[e._v("Currently it is possible to add custom properties to a Table Schema, but the lack of consensus about those properties restricts common tooling and wider adoption.")]),e._v(" "),a("h3",{attrs:{id:"use-cases"}},[a("a",{staticClass:"header-anchor",attrs:{href:"#use-cases"}},[e._v("#")]),e._v(" Use cases")]),e._v(" "),a("ul",[a("li",[e._v("Documentation: generating Markdown documentation from the schema itself is a useful use case, and contextual information (description, version, authors…) needs to be retrieved.")]),e._v(" "),a("li",[e._v("Cataloging: open data standardisation can be increased by improving Table Schemas shareability, for example by searching and categorising them (by keywords, countries, full-text…) in catalogs.")]),e._v(" "),a("li",[e._v("Machine readability: tools like Goodtables could use catalogs to access Table Schemas in order to help users validate tabular files against existing schemas. Metadata would be needed for tools to find and read those schemas.")])]),e._v(" "),a("h3",{attrs:{id:"specification-9"}},[a("a",{staticClass:"header-anchor",attrs:{href:"#specification-9"}},[e._v("#")]),e._v(" Specification")]),e._v(" "),a("p",[e._v("This pattern introduces the following properties to the Table Schema spec (using "),a("a",{attrs:{href:"https://github.com/frictionlessdata/specs/blob/master/schemas/dictionary/common.yml",target:"_blank",rel:"noopener noreferrer"}},[e._v("the Frictionless Data core dictionary"),a("OutboundLink")],1),e._v(" as much as possible):")]),e._v(" "),a("ul",[a("li",[a("code",[e._v("name")]),e._v(": An identifier string for this schema.")]),e._v(" "),a("li",[a("code",[e._v("title")]),e._v(": A human-readable title for this schema.")]),e._v(" "),a("li",[a("code",[e._v("description")]),e._v(": A text description for this schema.")]),e._v(" "),a("li",[a("code",[e._v("keywords")]),e._v(": The keyword(s) that describe this schema."),a("br"),e._v(" "),a("em",[e._v("Tags are useful to categorise and catalog schemas.")])]),e._v(" "),a("li",[a("code",[e._v("countryCode")]),e._v(": The ISO 3166-1 alpha-2 code for the country where this schema is primarily used."),a("br"),e._v(" "),a("em",[e._v("Since open data schemas are very country-specific, it’s useful to have this information in a structured way.")])]),e._v(" "),a("li",[a("code",[e._v("homepage")]),e._v(": The home on the web that is related to this schema.")]),e._v(" "),a("li",[a("code",[e._v("path")]),e._v(": A fully qualified URL for this schema."),a("br"),e._v(" "),a("em",[e._v("The direct path to the schema itself can be useful to help accessing it (i.e. machine readability).")])]),e._v(" "),a("li",[a("code",[e._v("image")]),e._v(": An image to represent this schema."),a("br"),e._v(" "),a("em",[e._v("An optional illustration can be useful for example in catalogs to differentiate schemas in a list.")])]),e._v(" "),a("li",[a("code",[e._v("licenses")]),e._v(": The license(s) under which this schema is published.")]),e._v(" "),a("li",[a("code",[e._v("resources")]),e._v(": Example tabular data resource(s) validated or invalidated against this schema."),a("br"),e._v(" "),a("em",[e._v("Oftentimes, schemas are shared with example resources to illustrate them, with valid or even invalid files (e.g. with constraint errors).")])]),e._v(" "),a("li",[a("code",[e._v("sources")]),e._v(": The source(s) used to created this schema."),a("br"),e._v(" "),a("em",[e._v("In some cases, schemas are created after a legal text or some draft specification in a human-readable document. In those cases, it’s useful to share them with the schema.")])]),e._v(" "),a("li",[a("code",[e._v("created")]),e._v(": The datetime on which this schema was created.")]),e._v(" "),a("li",[a("code",[e._v("lastModified")]),e._v(": The datetime on which this schema was last modified.")]),e._v(" "),a("li",[a("code",[e._v("version")]),e._v(": A unique version number for this schema.")]),e._v(" "),a("li",[a("code",[e._v("contributors")]),e._v(": The contributors to this schema.")])]),e._v(" "),a("h3",{attrs:{id:"example-schema"}},[a("a",{staticClass:"header-anchor",attrs:{href:"#example-schema"}},[e._v("#")]),e._v(" Example schema")]),e._v(" "),a("div",{staticClass:"language- extra-class"},[a("pre",{pre:!0,attrs:{class:"language-text"}},[a("code",[e._v('{\n "$schema": "https://specs.frictionlessdata.io/schemas/table-schema.json",\n "name": "irve",\n "title": "Infrastructures de recharge de véhicules électriques",\n "description": "Spécification du fichier d\'échange relatif aux données concernant la localisation géographique et les caractéristiques techniques des stations et des points de recharge pour véhicules électriques",\n "keywords": [\n "electric vehicle",\n "ev",\n "charging station",\n "mobility"\n ],\n "countryCode": "FR",\n "homepage": "https://github.com/etalab/schema-irve",\n "path": "https://github.com/etalab/schema-irve/raw/v1.0.1/schema.json",\n "image": "https://github.com/etalab/schema-irve/raw/v1.0.1/irve.png",\n "licenses": [\n {\n "title": "Creative Commons Zero v1.0 Universal",\n "name": "CC0-1.0",\n "path": "https://creativecommons.org/publicdomain/zero/1.0/"\n }\n ],\n "resources": [\n {\n "title": "Valid resource",\n "name": "exemple-valide",\n "path": "https://github.com/etalab/schema-irve/raw/v1.0.1/exemple-valide.csv"\n },\n {\n "title": "Invalid resource",\n "name": "exemple-invalide",\n "path": "https://github.com/etalab/schema-irve/raw/v1.0.1/exemple-invalide.csv"\n }\n ],\n "sources": [\n {\n "title": "Arrêté du 12 janvier 2017 relatif aux données concernant la localisation géographique et les caractéristiques techniques des stations et des points de recharge pour véhicules électriques",\n "path": "https://www.legifrance.gouv.fr/eli/arrete/2017/1/12/ECFI1634257A/jo/texte"\n }\n ],\n "created": "2018-06-29",\n "lastModified": "2019-05-06",\n "version": "1.0.1",\n "contributors": [\n {\n "title": "John Smith",\n "email": "john.smith@etalab.gouv.fr",\n "organization": "Etalab",\n "role": "author"\n },\n {\n "title": "Jane Doe",\n "email": "jane.doe@aol.com",\n "organization": "Civil Society Organization X",\n "role": "contributor"\n }\n ],\n "fields": [ ]\n}\n')])])]),a("h3",{attrs:{id:"implementations-7"}},[a("a",{staticClass:"header-anchor",attrs:{href:"#implementations-7"}},[e._v("#")]),e._v(" Implementations")]),e._v(" "),a("p",[e._v("The following links are actual examples already using this pattern, but not 100 % aligned with our proposal. The point is to make the Table Schema users converge towards a common pattern, before considering changing the spec.")]),e._v(" "),a("ul",[a("li",[e._v("@OpenDataFrance has initiated the creation of "),a("a",{attrs:{href:"http://git.opendatafrance.net/scdl/",target:"_blank",rel:"noopener noreferrer"}},[e._v("Table Schemas"),a("OutboundLink")],1),e._v(" to standardise common French open data datasets. "),a("a",{attrs:{href:"http://scdl.opendatafrance.net/",target:"_blank",rel:"noopener noreferrer"}},[e._v("Their Markdown documentation"),a("OutboundLink")],1),e._v(" is generated automatically from the schemas ("),a("a",{attrs:{href:"https://git.opendatafrance.net/validata/validata-doc-generator/",target:"_blank",rel:"noopener noreferrer"}},[e._v("using some scripts"),a("OutboundLink")],1),e._v("), including contextual information.")]),e._v(" "),a("li",[e._v("A tool called "),a("a",{attrs:{href:"https://go.validata.fr/",target:"_blank",rel:"noopener noreferrer"}},[e._v("Validata"),a("OutboundLink")],1),e._v(" was developed, based on Goodtables, to help French open data producers follow the schemas. It uses metadata from the schemas to present them.")]),e._v(" "),a("li",[e._v("@Etalab has launched "),a("a",{attrs:{href:"http://schema.data.gouv.fr/",target:"_blank",rel:"noopener noreferrer"}},[e._v("schema.data.gouv.fr"),a("OutboundLink")],1),e._v(", an official open data schema catalog, which is specific to France. "),a("a",{attrs:{href:"https://schema.data.gouv.fr/documentation/validation-schemas#validations-sp%C3%A9cifiques-au-format-table-schema",target:"_blank",rel:"noopener noreferrer"}},[e._v("It needs additional metadata in the schemas to validate them"),a("OutboundLink")],1),e._v(".")]),e._v(" "),a("li",[a("a",{attrs:{href:"https://github.com/etalab/schema-irve/blob/master/schema.json",target:"_blank",rel:"noopener noreferrer"}},[e._v("Example Table Schema"),a("OutboundLink")],1),e._v(" from @Etalab using metadata properties.")])]),e._v(" "),a("h2",{attrs:{id:"json-data-resources"}},[a("a",{staticClass:"header-anchor",attrs:{href:"#json-data-resources"}},[e._v("#")]),e._v(" JSON Data Resources")]),e._v(" "),a("h3",{attrs:{id:"overview-8"}},[a("a",{staticClass:"header-anchor",attrs:{href:"#overview-8"}},[e._v("#")]),e._v(" Overview")]),e._v(" "),a("p",[e._v("A simple format to describe a single structured JSON data resource. It includes support both for metadata such as author and title and a "),a("a",{attrs:{href:"https://json-schema.org/",target:"_blank",rel:"noopener noreferrer"}},[e._v("schema"),a("OutboundLink")],1),e._v(" to describe the data.")]),e._v(" "),a("h3",{attrs:{id:"introduction"}},[a("a",{staticClass:"header-anchor",attrs:{href:"#introduction"}},[e._v("#")]),e._v(" Introduction")]),e._v(" "),a("p",[e._v("A "),a("strong",[e._v("JSON Data Resource")]),e._v(" is a type of "),a("RouterLink",{attrs:{to:"/data-resource/"}},[e._v("Data Resource")]),e._v(" specialized for describing structured JSON data.")],1),e._v(" "),a("p",[e._v("JSON Data Resource extends "),a("RouterLink",{attrs:{to:"/data-resource/"}},[e._v("Data Resource")]),e._v(" in following key ways:")],1),e._v(" "),a("ul",[a("li",[e._v("The "),a("code",[e._v("schema")]),e._v(" property MUST follow the "),a("a",{attrs:{href:"https://json-schema.org/",target:"_blank",rel:"noopener noreferrer"}},[e._v("JSON Schema"),a("OutboundLink")],1),e._v(" specification,"),a("br"),e._v("\neither as a JSON object directly under the property, or a string referencing another"),a("br"),e._v("\nJSON document containing the JSON Schema")])]),e._v(" "),a("h3",{attrs:{id:"examples"}},[a("a",{staticClass:"header-anchor",attrs:{href:"#examples"}},[e._v("#")]),e._v(" Examples")]),e._v(" "),a("p",[e._v("A minimal JSON Data Resource, referencing external JSON documents, looks as follows.")]),e._v(" "),a("div",{staticClass:"language-javascript extra-class"},[a("pre",{pre:!0,attrs:{class:"language-javascript"}},[a("code",[a("span",{pre:!0,attrs:{class:"token comment"}},[e._v("// with data and a schema accessible via the local filesystem")]),e._v("\n"),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("{")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"profile"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"json-data-resource"')]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v(",")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"name"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"resource-name"')]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v(",")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"path"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("[")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"resource-path.json"')]),e._v(" "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("]")]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v(",")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"schema"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"jsonschema.json"')]),e._v("\n"),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("}")]),e._v("\n\n"),a("span",{pre:!0,attrs:{class:"token comment"}},[e._v("// with data accessible via http")]),e._v("\n"),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("{")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"profile"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"json-data-resource"')]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v(",")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"name"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"resource-name"')]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v(",")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"path"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("[")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"http://example.com/resource-path.json"')]),e._v(" "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("]")]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v(",")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"schema"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"http://example.com/jsonschema.json"')]),e._v("\n"),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("}")]),e._v("\n")])])]),a("p",[e._v("A minimal JSON Data Resource example using the data property to inline data looks as follows.")]),e._v(" "),a("div",{staticClass:"language-javascript extra-class"},[a("pre",{pre:!0,attrs:{class:"language-javascript"}},[a("code",[a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("{")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"profile"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"json-data-resource"')]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v(",")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"name"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"resource-name"')]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v(",")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"data"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("{")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"id"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token number"}},[e._v("1")]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v(",")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"first_name"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"Louise"')]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("}")]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v(",")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"schema"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("{")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"type"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"object"')]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v(",")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"required"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("[")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"id"')]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("]")]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v(",")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"properties"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("{")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"id"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("{")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"type"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"integer"')]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("}")]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v(",")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"first_name"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("{")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"type"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"string"')]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("}")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("}")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("}")]),e._v("\n"),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("}")]),e._v("\n")])])]),a("p",[e._v("A comprehensive JSON Data Resource example with all required, recommended and optional properties looks as follows.")]),e._v(" "),a("div",{staticClass:"language-javascript extra-class"},[a("pre",{pre:!0,attrs:{class:"language-javascript"}},[a("code",[a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("{")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"profile"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"json-data-resource"')]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v(",")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"name"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"solar-system"')]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v(",")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"path"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"http://example.com/solar-system.json"')]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v(",")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"title"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"The Solar System"')]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v(",")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"description"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"My favourite data about the solar system."')]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v(",")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"format"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"json"')]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v(",")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"mediatype"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"application/json"')]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v(",")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"encoding"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"utf-8"')]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v(",")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"bytes"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token number"}},[e._v("1")]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v(",")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"hash"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('""')]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v(",")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"schema"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("{")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"$schema"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"http://json-schema.org/draft-07/schema#"')]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v(",")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"type"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"object"')]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v(",")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"required"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("[")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"id"')]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("]")]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v(",")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"properties"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("{")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"id"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("{")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"type"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"integer"')]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("}")]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v(",")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"name"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("{")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"type"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"string"')]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("}")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"description"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("{")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"type"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"string"')]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("}")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("}")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("}")]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v(",")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"sources"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("[")]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("{")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"title"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"The Solar System - 2001"')]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v(",")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"path"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"http://example.com/solar-system-2001.json"')]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v(",")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"email"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('""')]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("}")]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("]")]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v(",")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"licenses"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("[")]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("{")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"name"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"CC-BY-4.0"')]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v(",")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"title"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"Creative Commons Attribution 4.0"')]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v(",")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"path"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"https://creativecommons.org/licenses/by/4.0/"')]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("}")]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("]")]),e._v("\n"),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("}")]),e._v("\n")])])]),a("h3",{attrs:{id:"specification-10"}},[a("a",{staticClass:"header-anchor",attrs:{href:"#specification-10"}},[e._v("#")]),e._v(" Specification")]),e._v(" "),a("p",[e._v("A JSON Data Resource MUST be a "),a("RouterLink",{attrs:{to:"/data-resource/"}},[e._v("Data Resource")]),e._v(", that is it MUST conform to the "),a("RouterLink",{attrs:{to:"/data-resource/"}},[e._v("Data Resource specification")]),e._v(".")],1),e._v(" "),a("p",[e._v("In addition:")]),e._v(" "),a("ul",[a("li",[e._v("The Data Resource "),a("code",[e._v("schema")]),e._v(" property MUST follow the "),a("a",{attrs:{href:"https://json-schema.org/",target:"_blank",rel:"noopener noreferrer"}},[e._v("JSON Schema"),a("OutboundLink")],1),e._v(" specification,"),a("br"),e._v("\neither as a JSON object directly under the property, or a string referencing another"),a("br"),e._v("\nJSON document containing the JSON Schema")])]),e._v(" "),a("ul",[a("li",[e._v("There "),a("code",[e._v("MUST")]),e._v(" be a "),a("code",[e._v("profile")]),e._v(" property with the value "),a("code",[e._v("json-data-resource")])])]),e._v(" "),a("ul",[a("li",[e._v("The data the Data Resource describes MUST, if non-inline, be a JSON file")])]),e._v(" "),a("h3",{attrs:{id:"json-file-requirements"}},[a("a",{staticClass:"header-anchor",attrs:{href:"#json-file-requirements"}},[e._v("#")]),e._v(" JSON file requirements")]),e._v(" "),a("p",[e._v("When "),a("code",[e._v('"format": "json"')]),e._v(", files must strictly follow the "),a("a",{attrs:{href:"https://www.json.org/",target:"_blank",rel:"noopener noreferrer"}},[e._v("JSON specification"),a("OutboundLink")],1),e._v(". Some implementations "),a("code",[e._v("MAY")]),e._v(" support "),a("code",[e._v('"format": "jsonc"')]),e._v(", allowing for non-standard single line and block comments ("),a("code",[e._v("//")]),e._v(" and "),a("code",[e._v("/* */")]),e._v(" respectively).")]),e._v(" "),a("h3",{attrs:{id:"implementations-8"}},[a("a",{staticClass:"header-anchor",attrs:{href:"#implementations-8"}},[e._v("#")]),e._v(" Implementations")]),e._v(" "),a("p",[e._v("None known.")]),e._v(" "),a("h2",{attrs:{id:"describing-data-package-catalogs-using-the-data-package-format"}},[a("a",{staticClass:"header-anchor",attrs:{href:"#describing-data-package-catalogs-using-the-data-package-format"}},[e._v("#")]),e._v(" Describing Data Package Catalogs using the Data Package Format")]),e._v(" "),a("h3",{attrs:{id:"overview-9"}},[a("a",{staticClass:"header-anchor",attrs:{href:"#overview-9"}},[e._v("#")]),e._v(" Overview")]),e._v(" "),a("p",[e._v("There are scenarios where one needs to describe a collection of data packages, such as when building an online registry, or when building a pipeline that ingests multiple datasets.")]),e._v(" "),a("p",[e._v("In these scenarios, the collection can be described using a “Catalog”, where each dataset is represented as a single resource which has:")]),e._v(" "),a("div",{staticClass:"language-json extra-class"},[a("pre",{pre:!0,attrs:{class:"language-json"}},[a("code",[a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("{")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token property"}},[e._v('"profile"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"data-package"')]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v(",")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token property"}},[e._v('"format"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"json"')]),e._v("\n"),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("}")]),e._v("\n")])])]),a("h3",{attrs:{id:"specification-11"}},[a("a",{staticClass:"header-anchor",attrs:{href:"#specification-11"}},[e._v("#")]),e._v(" Specification")]),e._v(" "),a("p",[e._v("The Data Package Catalog builds directly on the Data Package specification. Thus a Data Package Catalog "),a("code",[e._v("MUST")]),e._v(" be a Data Package and conform to the "),a("RouterLink",{attrs:{to:"/data-package/"}},[e._v("Data Package specification")]),e._v(".")],1),e._v(" "),a("p",[e._v("The Data Package Catalog has the following requirements over and above those imposed by Data Package:")]),e._v(" "),a("ul",[a("li",[e._v("There "),a("code",[e._v("MUST")]),e._v(" be a "),a("code",[e._v("profile")]),e._v(" property with the value "),a("code",[e._v("data-package-catalog")]),e._v(", or a "),a("code",[e._v("profile")]),e._v(" that extends it")]),e._v(" "),a("li",[e._v("Each resource "),a("code",[e._v("MUST")]),e._v(" also be a Data Package")])]),e._v(" "),a("h4",{attrs:{id:"examples-2"}},[a("a",{staticClass:"header-anchor",attrs:{href:"#examples-2"}},[e._v("#")]),e._v(" Examples")]),e._v(" "),a("p",[e._v("A generic package catalog:")]),e._v(" "),a("div",{staticClass:"language-json extra-class"},[a("pre",{pre:!0,attrs:{class:"language-json"}},[a("code",[a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("{")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token property"}},[e._v('"profile"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"data-package-catalog"')]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v(",")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token property"}},[e._v('"name"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"climate-change-packages"')]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v(",")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token property"}},[e._v('"resources"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("[")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("{")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token property"}},[e._v('"profile"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"json-data-package"')]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v(",")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token property"}},[e._v('"format"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"json"')]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v(",")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token property"}},[e._v('"name"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"beacon-network-description"')]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v(",")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token property"}},[e._v('"path"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"https://http://beacon.berkeley.edu/hypothetical_deployment_description.json"')]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("}")]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v(",")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("{")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token property"}},[e._v('"profile"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"tabular-data-package"')]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v(",")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token property"}},[e._v('"format"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"json"')]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v(",")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token property"}},[e._v('"path"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"https://pkgstore.datahub.io/core/co2-ppm/10/datapackage.json"')]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("}")]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v(",")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("{")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token property"}},[e._v('"profile"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"tabular-data-package"')]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v(",")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token property"}},[e._v('"name"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"co2-fossil-global"')]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v(",")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token property"}},[e._v('"format"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"json"')]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v(",")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token property"}},[e._v('"path"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"https://pkgstore.datahub.io/core/co2-fossil-global/11/datapackage.json"')]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("}")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("]")]),e._v("\n"),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("}")]),e._v("\n")])])]),a("p",[e._v("A minimal tabular data catalog:")]),e._v(" "),a("div",{staticClass:"language-json extra-class"},[a("pre",{pre:!0,attrs:{class:"language-json"}},[a("code",[a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("{")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token property"}},[e._v('"profile"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"tabular-data-package-catalog"')]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v(",")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token property"}},[e._v('"name"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"datahub-climate-change-packages"')]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v(",")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token property"}},[e._v('"resources"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("[")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("{")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token property"}},[e._v('"path"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"https://pkgstore.datahub.io/core/co2-ppm/10/datapackage.json"')]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("}")]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v(",")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("{")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token property"}},[e._v('"name"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"co2-fossil-global"')]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v(",")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token property"}},[e._v('"path"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"https://pkgstore.datahub.io/core/co2-fossil-global/11/datapackage.json"')]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("}")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("]")]),e._v("\n"),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("}")]),e._v("\n")])])]),a("p",[e._v("Data packages can also be declared inline in the data catalog:")]),e._v(" "),a("div",{staticClass:"language-json extra-class"},[a("pre",{pre:!0,attrs:{class:"language-json"}},[a("code",[a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("{")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token property"}},[e._v('"profile"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"tabular-data-package-catalog"')]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v(",")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token property"}},[e._v('"name"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"my-data-catalog"')]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v(",")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token property"}},[e._v('"resources"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("[")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("{")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token property"}},[e._v('"profile"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"tabular-data-package"')]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v(",")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token property"}},[e._v('"name"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"my-dataset"')]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v(",")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token comment"}},[e._v("// here we list the data files in this dataset")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token property"}},[e._v('"resources"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("[")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("{")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token property"}},[e._v('"profile"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"tabular-data-resource"')]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v(",")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token property"}},[e._v('"name"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"resource-name"')]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v(",")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token property"}},[e._v('"data"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("[")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("{")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token property"}},[e._v('"id"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token number"}},[e._v("1")]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v(",")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token property"}},[e._v('"first_name"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"Louise"')]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("}")]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v(",")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("{")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token property"}},[e._v('"id"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token number"}},[e._v("2")]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v(",")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token property"}},[e._v('"first_name"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"Julia"')]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("}")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("]")]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v(",")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token property"}},[e._v('"schema"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("{")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token property"}},[e._v('"fields"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("[")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("{")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token property"}},[e._v('"name"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"id"')]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v(",")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token property"}},[e._v('"type"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"integer"')]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("}")]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v(",")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("{")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token property"}},[e._v('"name"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"first_name"')]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v(",")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token property"}},[e._v('"type"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"string"')]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("}")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("]")]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v(",")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token property"}},[e._v('"primaryKey"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"id"')]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("}")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("}")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("]")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("}")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("]")]),e._v("\n"),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("}")]),e._v("\n")])])]),a("h3",{attrs:{id:"implementations-9"}},[a("a",{staticClass:"header-anchor",attrs:{href:"#implementations-9"}},[e._v("#")]),e._v(" Implementations")]),e._v(" "),a("p",[e._v("None known.")]),e._v(" "),a("h2",{attrs:{id:"table-schema-unique-constraints"}},[a("a",{staticClass:"header-anchor",attrs:{href:"#table-schema-unique-constraints"}},[e._v("#")]),e._v(" Table Schema: Unique constraints")]),e._v(" "),a("h3",{attrs:{id:"overview-10"}},[a("a",{staticClass:"header-anchor",attrs:{href:"#overview-10"}},[e._v("#")]),e._v(" Overview")]),e._v(" "),a("p",[e._v("A "),a("code",[e._v("primaryKey")]),e._v(" uniquely identifies each row in a table. Per SQL standards, it"),a("br"),e._v("\ncannot contain "),a("code",[e._v("null")]),e._v(" values. This pattern implements the SQL UNIQUE constraint"),a("br"),e._v("\nby introducing a "),a("code",[e._v("uniqueKeys")]),e._v(" array, defining one or more row uniqueness"),a("br"),e._v("\nconstraints which do support "),a("code",[e._v("null")]),e._v(" values. An additional "),a("code",[e._v("uniqueNulls")]),e._v(" property"),a("br"),e._v("\ncontrols how "),a("code",[e._v("null")]),e._v(" values are to be treated in unique constraints.")]),e._v(" "),a("h3",{attrs:{id:"specification-12"}},[a("a",{staticClass:"header-anchor",attrs:{href:"#specification-12"}},[e._v("#")]),e._v(" Specification")]),e._v(" "),a("h4",{attrs:{id:"uniquekeys-add"}},[a("a",{staticClass:"header-anchor",attrs:{href:"#uniquekeys-add"}},[e._v("#")]),e._v(" "),a("code",[e._v("uniqueKeys")]),e._v(" (add)")]),e._v(" "),a("p",[e._v("The "),a("code",[e._v("uniqueKeys")]),e._v(" property, if present, "),a("code",[e._v("MUST")]),e._v(" be an array. Each entry"),a("br"),e._v("\n("),a("code",[e._v("uniqueKey")]),e._v(") in the array "),a("code",[e._v("MUST")]),e._v(" be a string or array (structured as per"),a("br"),e._v(" "),a("code",[e._v("primaryKey")]),e._v(") specifying the resource field or fields required to be unique for"),a("br"),e._v("\neach row in the table.")]),e._v(" "),a("h4",{attrs:{id:"uniquenulls-add"}},[a("a",{staticClass:"header-anchor",attrs:{href:"#uniquenulls-add"}},[e._v("#")]),e._v(" "),a("code",[e._v("uniqueNulls")]),e._v(" (add)")]),e._v(" "),a("p",[e._v("The "),a("code",[e._v("uniqueNulls")]),e._v(" property is a boolean that dictates how "),a("code",[e._v("null")]),e._v(" values should"),a("br"),e._v("\nbe treated by all unique constraints set on a resource.")]),e._v(" "),a("ul",[a("li",[e._v("If "),a("code",[e._v("true")]),e._v(" (the default), "),a("code",[e._v("null")]),e._v(" values are treated as unique (per most SQL"),a("br"),e._v("\ndatabases). By this definition, "),a("code",[e._v("1, null, null")]),e._v(" is UNIQUE.")]),e._v(" "),a("li",[e._v("If "),a("code",[e._v("false")]),e._v(", "),a("code",[e._v("null")]),e._v(" values are treated like any other value (per Microsoft SQL"),a("br"),e._v("\nServer, Python pandas, R data.frame, Google Sheets). By this definition, "),a("code",[e._v("1, null, null")]),e._v(" is NOT UNIQUE.")])]),e._v(" "),a("h4",{attrs:{id:"foreignkeys-edit"}},[a("a",{staticClass:"header-anchor",attrs:{href:"#foreignkeys-edit"}},[e._v("#")]),e._v(" "),a("code",[e._v("foreignKeys")]),e._v(" (edit)")]),e._v(" "),a("p",[e._v("Per SQL standards, "),a("code",[e._v("null")]),e._v(" values are permitted in both the local and reference"),a("br"),e._v("\nkeys of a foreign key. However, reference keys "),a("code",[e._v("MUST")]),e._v(" be unique and are"),a("br"),e._v("\ntherefore equivalent to a "),a("code",[e._v("uniqueKey")]),e._v(" set on the reference resource (the meaning"),a("br"),e._v("\nof which is determined by the reference "),a("code",[e._v("uniqueNulls")]),e._v(").")]),e._v(" "),a("p",[e._v("Furthermore, per SQL standards, the local key "),a("code",[e._v("MAY")]),e._v(" contain keys with field"),a("br"),e._v("\nvalues not present in the reference key if and only if at least one of the"),a("br"),e._v("\nfields is locally "),a("code",[e._v("null")]),e._v(". For example, "),a("code",[e._v("(1, null)")]),e._v(" is permitted locally even if"),a("br"),e._v("\nthe reference is "),a("code",[e._v("[(2, 1), (3, 1)]")]),e._v(". This behavior is the same regardless of the"),a("br"),e._v("\nvalue of "),a("code",[e._v("uniqueNulls")]),e._v(".")]),e._v(" "),a("h3",{attrs:{id:"examples-3"}},[a("a",{staticClass:"header-anchor",attrs:{href:"#examples-3"}},[e._v("#")]),e._v(" Examples")]),e._v(" "),a("h4",{attrs:{id:"null-in-unique-constraints"}},[a("a",{staticClass:"header-anchor",attrs:{href:"#null-in-unique-constraints"}},[e._v("#")]),e._v(" "),a("code",[e._v("null")]),e._v(" in unique constraints")]),e._v(" "),a("table",[a("thead",[a("tr",[a("th",[e._v("a")]),e._v(" "),a("th",[e._v("b")]),e._v(" "),a("th",[e._v("c")]),e._v(" "),a("th",[e._v("d")])])]),e._v(" "),a("tbody",[a("tr",[a("td",[e._v("1")]),e._v(" "),a("td",[e._v("1")]),e._v(" "),a("td",[e._v("1")]),e._v(" "),a("td",[e._v("1")])]),e._v(" "),a("tr",[a("td",[e._v("2")]),e._v(" "),a("td",[e._v("2")]),e._v(" "),a("td",[a("code",[e._v("null")])]),e._v(" "),a("td",[e._v("2")])]),e._v(" "),a("tr",[a("td",[e._v("3")]),e._v(" "),a("td",[e._v("2")]),e._v(" "),a("td",[a("code",[e._v("null")])]),e._v(" "),a("td",[a("code",[e._v("null")])])])])]),e._v(" "),a("p",[e._v("The above table meets the following primary key and two unique key constraints:")]),e._v(" "),a("div",{staticClass:"language-json extra-class"},[a("pre",{pre:!0,attrs:{class:"language-json"}},[a("code",[a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("{")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token property"}},[e._v('"primaryKey"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("[")]),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"a"')]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("]")]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v(",")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token property"}},[e._v('"uniqueKeys"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("[")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("[")]),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"b"')]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v(",")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"c"')]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("]")]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v(",")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("[")]),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"c"')]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v(",")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"d"')]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("]")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("]")]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v(",")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token property"}},[e._v('"uniqueNulls"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token boolean"}},[e._v("true")]),e._v("\n"),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("}")]),e._v("\n")])])]),a("p",[e._v("The primary key "),a("code",[e._v("(a)")]),e._v(" only contains unique, non-"),a("code",[e._v("null")]),e._v(" values. In contrast, the"),a("br"),e._v("\nunique keys can contain "),a("code",[e._v("null")]),e._v(" values. Although unique key "),a("code",[e._v("(b, c)")]),e._v(" contains two"),a("br"),e._v("\nidentical keys "),a("code",[e._v("(2, null)")]),e._v(", this is permitted because "),a("code",[e._v("uniqueNulls: true")]),a("br"),e._v("\nspecifies that "),a("code",[e._v("null")]),e._v(" values are unique. This behavior is consistent with the"),a("br"),e._v("\nUNIQUE constraint of PostgreSQL and most other SQL implementations, as"),a("br"),e._v("\nillustrated by this"),a("br"),e._v(" "),a("a",{attrs:{href:"https://dbfiddle.uk/?rdbms=postgres_11&fiddle=34cab8ba7d74b488d215a96f7e83c096",target:"_blank",rel:"noopener noreferrer"}},[e._v("dbfiddle"),a("OutboundLink")],1),e._v("."),a("br"),e._v("\nThe same keys would be considered duplicates if "),a("code",[e._v("uniqueNulls: false")]),e._v(", consistent"),a("br"),e._v("\nwith the UNIQUE constraint of Microsoft SQL Server, as illustrated by this"),a("br"),e._v(" "),a("a",{attrs:{href:"https://dbfiddle.uk/?rdbms=sqlserver_2019l&fiddle=34cab8ba7d74b488d215a96f7e83c096",target:"_blank",rel:"noopener noreferrer"}},[e._v("dbfiddle"),a("OutboundLink")],1),e._v(".")]),e._v(" "),a("h4",{attrs:{id:"setting-unique-constraints"}},[a("a",{staticClass:"header-anchor",attrs:{href:"#setting-unique-constraints"}},[e._v("#")]),e._v(" Setting unique constraints")]),e._v(" "),a("p",[e._v("For a given resource, unique constraints can be set for one field using a"),a("br"),e._v("\nfield’s "),a("code",[e._v("unique")]),e._v(" constraint, for one or multiple fields using a "),a("code",[e._v("uniqueKey")]),e._v(", and"),a("br"),e._v("\nfor one or multiple fields using a "),a("code",[e._v("foreignKey")]),e._v(" referencing the resource. Each"),a("br"),e._v("\nof the following examples set a unique constraint on field "),a("code",[e._v("a")]),e._v(":")]),e._v(" "),a("p",[a("strong",[e._v("Field "),a("code",[e._v("constraints")])])]),e._v(" "),a("div",{staticClass:"language-json extra-class"},[a("pre",{pre:!0,attrs:{class:"language-json"}},[a("code",[a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("{")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token property"}},[e._v('"fields"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("[")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("{")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token property"}},[e._v('"name"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"a"')]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v(",")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token property"}},[e._v('"constraints"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("{")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token property"}},[e._v('"unique"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token boolean"}},[e._v("true")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("}")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("}")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("]")]),e._v("\n"),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("}")]),e._v("\n")])])]),a("p",[a("strong",[a("code",[e._v("uniqueKeys")])])]),e._v(" "),a("div",{staticClass:"language-json extra-class"},[a("pre",{pre:!0,attrs:{class:"language-json"}},[a("code",[a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("{")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token property"}},[e._v('"uniqueKeys"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("[")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"a"')]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("]")]),e._v("\n"),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("}")]),e._v("\n")])])]),a("p",[a("strong",[a("code",[e._v("foreignKeys")])])]),e._v(" "),a("div",{staticClass:"language-json extra-class"},[a("pre",{pre:!0,attrs:{class:"language-json"}},[a("code",[a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("{")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token property"}},[e._v('"foreignKeys"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("[")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("{")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token property"}},[e._v('"fields"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"a"')]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v(",")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token property"}},[e._v('"reference"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("{")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token property"}},[e._v('"resource"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('""')]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v(",")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token property"}},[e._v('"fields"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"a"')]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("}")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("}")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("]")]),e._v("\n"),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("}")]),e._v("\n")])])]),a("h3",{attrs:{id:"implementations-10"}},[a("a",{staticClass:"header-anchor",attrs:{href:"#implementations-10"}},[e._v("#")]),e._v(" Implementations")]),e._v(" "),a("p",[e._v("None known.")]),e._v(" "),a("h2",{attrs:{id:"describing-files-inside-a-compressed-file-such-as-zip"}},[a("a",{staticClass:"header-anchor",attrs:{href:"#describing-files-inside-a-compressed-file-such-as-zip"}},[e._v("#")]),e._v(" Describing files inside a compressed file such as Zip")]),e._v(" "),a("h3",{attrs:{id:"overview-11"}},[a("a",{staticClass:"header-anchor",attrs:{href:"#overview-11"}},[e._v("#")]),e._v(" Overview")]),e._v(" "),a("p",[e._v("Some datasets need to contain a Zip file (or tar, other formats) containing a"),a("br"),e._v("\nset of files.")]),e._v(" "),a("p",[e._v("This might happen for practical reasons (datasets containing thousands of files)"),a("br"),e._v("\nor for technical limitations (for example, currently Zenodo doesn’t support subdirectories and"),a("br"),e._v("\ndatasets might need subdirectory structures to be useful).")]),e._v(" "),a("h3",{attrs:{id:"implementations-11"}},[a("a",{staticClass:"header-anchor",attrs:{href:"#implementations-11"}},[e._v("#")]),e._v(" Implementations")]),e._v(" "),a("p",[e._v("There are no known implementations at present.")]),e._v(" "),a("h3",{attrs:{id:"specification-13"}},[a("a",{staticClass:"header-anchor",attrs:{href:"#specification-13"}},[e._v("#")]),e._v(" Specification")]),e._v(" "),a("p",[e._v("The "),a("code",[e._v("resources")]),e._v(" in a "),a("code",[e._v("data-package")]),e._v(" can contain “recursive resources”: identifying"),a("br"),e._v("\na new resource.")]),e._v(" "),a("h3",{attrs:{id:"example"}},[a("a",{staticClass:"header-anchor",attrs:{href:"#example"}},[e._v("#")]),e._v(" Example")]),e._v(" "),a("div",{staticClass:"language-json extra-class"},[a("pre",{pre:!0,attrs:{class:"language-json"}},[a("code",[a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("{")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token property"}},[e._v('"profile"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"data-package"')]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v(",")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token property"}},[e._v('"resources"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("[")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("{")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token property"}},[e._v('"path"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"https://zenodo.org/record/3247384/files/Sea-Bird_Processed_Data.zip"')]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v(",")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token property"}},[e._v('"format"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"zip"')]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v(",")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token property"}},[e._v('"mediatype"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"application/zip"')]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v(",")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token property"}},[e._v('"bytes"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"294294242424"')]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v(",")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token property"}},[e._v('"hash"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"a27063c614c183b502e5c03bd9c8931b"')]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v(",")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token property"}},[e._v('"resources"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("[")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("{")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token property"}},[e._v('"path"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"file_name.csv"')]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v(",")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token property"}},[e._v('"format"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"csv"')]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v(",")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token property"}},[e._v('"mediatype"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"text/csv"')]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v(",")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token property"}},[e._v('"bytes"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token number"}},[e._v("242421")]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v(",")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token property"}},[e._v('"hash"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"0300048878bb9b5804a1f62869d296bc"')]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v(",")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token property"}},[e._v('"profile"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"tabular-data-resource"')]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v(",")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token property"}},[e._v('"schema"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"tableschema.json"')]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("}")]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v(",")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("{")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token property"}},[e._v('"path"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"directory/file_name2.csv"')]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v(",")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token property"}},[e._v('"format"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"csv"')]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v(",")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token property"}},[e._v('"mediatype"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"text/csv"')]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v(",")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token property"}},[e._v('"bytes"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token number"}},[e._v("2424213")]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v(",")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token property"}},[e._v('"hash"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"ff9435e0ee350efbe8a4a8779a47caaa"')]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v(",")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token property"}},[e._v('"profile"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"tabular-data-resource"')]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v(",")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token property"}},[e._v('"schema"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"tableschema.json"')]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("}")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("]")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("}")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("]")]),e._v("\n"),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("}")]),e._v("\n")])])]),a("p",[e._v("For a "),a("code",[e._v(".tar.gz")]),e._v(" it would be the same changing the "),a("code",[e._v('"format"')]),e._v(" and the"),a("br"),e._v(" "),a("code",[e._v('"mediatype"')]),e._v(".")]),e._v(" "),a("h3",{attrs:{id:"types-of-files"}},[a("a",{staticClass:"header-anchor",attrs:{href:"#types-of-files"}},[e._v("#")]),e._v(" Types of files")]),e._v(" "),a("p",[e._v("Support for "),a("code",[e._v("Zip")]),e._v(" and "),a("code",[e._v("tar.gz")]),e._v(" might be enough: hopefully everything can be"),a("br"),e._v("\nre-packaged using these formats.")]),e._v(" "),a("p",[e._v("To keep the implementation and testing testing: only one recursive level is"),a("br"),e._v("\npossible. A "),a("code",[e._v("resource")]),e._v(" can list "),a("code",[e._v("resources")]),e._v(" inside (like in the example). But"),a("br"),e._v("\nthe inner resources cannot contain resources again.")]),e._v(" "),a("h2",{attrs:{id:"missing-values-per-field"}},[a("a",{staticClass:"header-anchor",attrs:{href:"#missing-values-per-field"}},[e._v("#")]),e._v(" Missing values per field")]),e._v(" "),a("h3",{attrs:{id:"overview-12"}},[a("a",{staticClass:"header-anchor",attrs:{href:"#overview-12"}},[e._v("#")]),e._v(" Overview")]),e._v(" "),a("p",[e._v("Characters representing missing values in a table can be defined for all fields in a "),a("a",{attrs:{href:"http://frictionlessdata.io/specs/tabular-data-resource/",target:"_blank",rel:"noopener noreferrer"}},[e._v("Tabular Data Resource"),a("OutboundLink")],1),e._v(" using the "),a("a",{attrs:{href:"http://frictionlessdata.io/specs/table-schema/#missing-values",target:"_blank",rel:"noopener noreferrer"}},[a("code",[e._v("missingValues")]),a("OutboundLink")],1),e._v(" property in a Table Schema. Values that match the "),a("code",[e._v("missingValues")]),e._v(" are treated as "),a("code",[e._v("null")]),e._v(".")]),e._v(" "),a("p",[e._v("The Missing values per field pattern allows different missing values to be specified for each field in a Table Schema. If not specified, each field inherits from values assigned to "),a("code",[e._v("missingValues")]),e._v(" at the Tabular Data Resource level.")]),e._v(" "),a("p",[e._v("For example, this data…")]),e._v(" "),a("table",[a("thead",[a("tr",[a("th",[e._v("item")]),e._v(" "),a("th",[e._v("description")]),e._v(" "),a("th",[e._v("price")])])]),e._v(" "),a("tbody",[a("tr",[a("td",[e._v("1")]),e._v(" "),a("td",[e._v("Apple")]),e._v(" "),a("td",[e._v("0.99")])]),e._v(" "),a("tr",[a("td",[e._v("tba")]),e._v(" "),a("td",[e._v("Banana")]),e._v(" "),a("td",[e._v("-1")])]),e._v(" "),a("tr",[a("td",[e._v("3")]),e._v(" "),a("td",[e._v("n/a")]),e._v(" "),a("td",[e._v("1.20")])])])]),e._v(" "),a("p",[e._v("…using this Table Schema…")]),e._v(" "),a("div",{staticClass:"language-javascript extra-class"},[a("pre",{pre:!0,attrs:{class:"language-javascript"}},[a("code",[a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"schema"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("{")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"fields"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("[")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("{")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"name"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"item"')]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v(",")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"title"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"An inventory item number"')]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v(",")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"type"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"integer"')]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("}")]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v(",")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("{")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"name"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"description"')]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v(",")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"title"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"item description"')]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v(",")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"type"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"string"')]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v(",")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"missingValues"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("[")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"n/a"')]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("]")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("}")]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v(",")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("{")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"name"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"price"')]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v(",")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"title"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"cost price"')]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v(",")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"type"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"number"')]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v(",")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"missingValues"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("[")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"-1"')]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("]")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("}")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("]")]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v(",")]),e._v("\n "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"missingValues"')]),a("span",{pre:!0,attrs:{class:"token operator"}},[e._v(":")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("[")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('"tba"')]),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v(",")]),e._v(" "),a("span",{pre:!0,attrs:{class:"token string"}},[e._v('""')]),e._v(" "),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("]")]),e._v("\n"),a("span",{pre:!0,attrs:{class:"token punctuation"}},[e._v("}")]),e._v("\n")])])]),a("p",[e._v("…would be interpreted as…")]),e._v(" "),a("table",[a("thead",[a("tr",[a("th",[e._v("item")]),e._v(" "),a("th",[e._v("description")]),e._v(" "),a("th",[e._v("price")])])]),e._v(" "),a("tbody",[a("tr",[a("td",[e._v("1")]),e._v(" "),a("td",[e._v("Apple")]),e._v(" "),a("td",[e._v("0.99")])]),e._v(" "),a("tr",[a("td",[a("code",[e._v("null")])]),e._v(" "),a("td",[e._v("Banana")]),e._v(" "),a("td",[a("code",[e._v("null")])])]),e._v(" "),a("tr",[a("td",[e._v("3")]),e._v(" "),a("td",[a("code",[e._v("null")])]),e._v(" "),a("td",[e._v("1.20")])])])]),e._v(" "),a("h3",{attrs:{id:"specification-14"}},[a("a",{staticClass:"header-anchor",attrs:{href:"#specification-14"}},[e._v("#")]),e._v(" Specification")]),e._v(" "),a("p",[e._v("A field MAY have a "),a("code",[e._v("missingValues")]),e._v(" property that MUST be an "),a("code",[e._v("array")]),e._v(" where each entry is a "),a("code",[e._v("string")]),e._v(". If not specified, each field inherits from the values assigned to "),a("a",{attrs:{href:"http://frictionlessdata.io/specs/table-schema/#missing-values",target:"_blank",rel:"noopener noreferrer"}},[a("code",[e._v("missingValues")]),a("OutboundLink")],1),e._v(" at the Tabular Data Resource level.")]),e._v(" "),a("h3",{attrs:{id:"implementations-12"}},[a("a",{staticClass:"header-anchor",attrs:{href:"#implementations-12"}},[e._v("#")]),e._v(" Implementations")]),e._v(" "),a("p",[e._v("None known.")]),e._v(" "),a("h2",{attrs:{id:"table-schema-enum-labels-and-ordering"}},[a("a",{staticClass:"header-anchor",attrs:{href:"#table-schema-enum-labels-and-ordering"}},[e._v("#")]),e._v(" Table Schema: Enum labels and ordering")]),e._v(" "),a("h3",{attrs:{id:"overview-13"}},[a("a",{staticClass:"header-anchor",attrs:{href:"#overview-13"}},[e._v("#")]),e._v(" Overview")]),e._v(" "),a("p",[e._v("Many software packages for manipulating and analyzing tabular data have special"),a("br"),e._v("\nfeatures for working with categorical variables. These include:")]),e._v(" "),a("ul",[a("li",[e._v("Value labels or formats ("),a("a",{attrs:{href:"https://www.stata.com/manuals13/dlabel.pdf",target:"_blank",rel:"noopener noreferrer"}},[e._v("Stata"),a("OutboundLink")],1),e._v(","),a("br"),e._v(" "),a("a",{attrs:{href:"https://documentation.sas.com/doc/en/pgmsascdc/9.4_3.5/proc/p1upn25lbfo6mkn1wncu4dyh9q91.htm",target:"_blank",rel:"noopener noreferrer"}},[e._v("SAS"),a("OutboundLink")],1),a("br"),e._v("\nand "),a("a",{attrs:{href:"https://www.ibm.com/docs/en/spss-statistics/beta?topic=data-adding-value-labels",target:"_blank",rel:"noopener noreferrer"}},[e._v("SPSS"),a("OutboundLink")],1),e._v(")")]),e._v(" "),a("li",[a("a",{attrs:{href:"https://pandas.pydata.org/docs/user_guide/categorical.html",target:"_blank",rel:"noopener noreferrer"}},[e._v("Categoricals (Pandas)"),a("OutboundLink")],1)]),e._v(" "),a("li",[a("a",{attrs:{href:"https://www.stat.berkeley.edu/~s133/factors.html",target:"_blank",rel:"noopener noreferrer"}},[e._v("Factors ®"),a("OutboundLink")],1)]),e._v(" "),a("li",[a("a",{attrs:{href:"https://dataframes.juliadata.org/stable/man/categorical/",target:"_blank",rel:"noopener noreferrer"}},[e._v("CategoricalVectors (Julia)"),a("OutboundLink")],1)])]),e._v(" "),a("p",[e._v("These features can result in more efficient storage and faster runtime"),a("br"),e._v("\nperformance, but more importantly, facilitate analysis by indicating that a"),a("br"),e._v("\nvariable should be treated as categorical and by permitting the logical order"),a("br"),e._v("\nof the categories to differ from their lexical order. And in the case of value"),a("br"),e._v("\nlabels, they permit the analyst to work with variables in numeric form (e.g.,"),a("br"),e._v("\nin expressions, when fitting models) while generating output (e.g., tables,"),a("br"),e._v("\nplots) that is labeled with informative strings.")]),e._v(" "),a("p",[e._v("While these features are of limited use in some disciplines, others rely"),a("br"),e._v("\nheavily on them (e.g., social sciences, epidemiology, clinical research,"),a("br"),e._v("\netc.). Thus, before these disciplines can begin to use Frictionless in a"),a("br"),e._v("\nmeaningful way, both the standards and the software tools need to support"),a("br"),e._v("\nthese features. This pattern addresses necessary extensions to the"),a("br"),e._v(" "),a("a",{attrs:{href:"https://specs.frictionlessdata.io//table-schema/",target:"_blank",rel:"noopener noreferrer"}},[e._v("Table Schema"),a("OutboundLink")],1),e._v(".")]),e._v(" "),a("h3",{attrs:{id:"principles"}},[a("a",{staticClass:"header-anchor",attrs:{href:"#principles"}},[e._v("#")]),e._v(" Principles")]),e._v(" "),a("p",[e._v("Before describing the proposed extensions, here are the principles on which"),a("br"),e._v("\nthey are based:")]),e._v(" "),a("ol",[a("li",[e._v("Extensions should be software agnostic (i.e., no additions to the official"),a("br"),e._v("\nschema targeted toward a specific piece of software). While the extensions"),a("br"),e._v("\nare intended to support the use of features not available in all software,"),a("br"),e._v("\nthe resulting data package should continue to work as well as possible with"),a("br"),e._v("\nsoftware that does not have those features.")]),e._v(" "),a("li",[e._v("Related to (1), extensions should only include metadata that describe the"),a("br"),e._v("\ndata themselves—not instructions for what a specific software package should"),a("br"),e._v("\ndo with the data. Users who want to include the latter may do so within"),a("br"),e._v("\na sub-namespace such as "),a("code",[e._v("custom")]),e._v(" (e.g., see Issues "),a("a",{attrs:{href:"https://github.com/frictionlessdata/specs/issues/103",target:"_blank",rel:"noopener noreferrer"}},[e._v("#103"),a("OutboundLink")],1),a("br"),e._v("\nand "),a("a",{attrs:{href:"https://github.com/frictionlessdata/specs/issues/663",target:"_blank",rel:"noopener noreferrer"}},[e._v("#663"),a("OutboundLink")],1),e._v(").")]),e._v(" "),a("li",[e._v("Extensions must be backward compatible (i.e., not break existing tools,"),a("br"),e._v("\nworkflows, etc. for working with Frictionless packages).")])]),e._v(" "),a("p",[e._v("It is worth emphasizing that the scope of the proposed extensions is strictly"),a("br"),e._v("\nlimited to the information necessary to make use of the features for working"),a("br"),e._v("\nwith categorical data provided by the software packages listed above. Previous"),a("br"),e._v("\ndiscussions of this issue have occasionally included references to additional"),a("br"),e._v("\nvariable-level metadata (e.g., multiple sets of category labels such as both"),a("br"),e._v("\n“short labels” and longer “descriptions”, or links to common data elements,"),a("br"),e._v("\ncontrolled vocabularies or rdfTypes). While these additional metadata are"),a("br"),e._v("\nundoubtedly useful, we speculate that the large majority of users who would"),a("br"),e._v("\nbenefit from the extensions propopsed here would not have and/or utilize such"),a("br"),e._v("\ninformation, and therefore argue that these should be considered under a"),a("br"),e._v("\nseparate proposal.")]),e._v(" "),a("h3",{attrs:{id:"implementations-13"}},[a("a",{staticClass:"header-anchor",attrs:{href:"#implementations-13"}},[e._v("#")]),e._v(" Implementations")]),e._v(" "),a("p",[e._v("Our proposal to add a field-specific "),a("code",[e._v("enumOrdered")]),e._v(" property has been raised"),a("br"),e._v(" "),a("a",{attrs:{href:"https://github.com/frictionlessdata/specs/issues/739",target:"_blank",rel:"noopener noreferrer"}},[e._v("here"),a("OutboundLink")],1),e._v(" and"),a("br"),e._v(" "),a("a",{attrs:{href:"https://github.com/frictionlessdata/specs/issues/156",target:"_blank",rel:"noopener noreferrer"}},[e._v("here"),a("OutboundLink")],1),e._v(".")]),e._v(" "),a("p",[e._v("Discussions regarding supporting software providing features for working with"),a("br"),e._v("\ncategorical variables appear in the following GitHub issues:")]),e._v(" "),a("ul",[a("li",[a("a",{attrs:{href:"https://github.com/frictionlessdata/specs/issues/156",target:"_blank",rel:"noopener noreferrer"}},[e._v("https://github.com/frictionlessdata/specs/issues/156"),a("OutboundLink")],1)]),e._v(" "),a("li",[a("a",{attrs:{href:"https://github.com/frictionlessdata/specs/issues/739",target:"_blank",rel:"noopener noreferrer"}},[e._v("https://github.com/frictionlessdata/specs/issues/739"),a("OutboundLink")],1)])]),e._v(" "),a("p",[e._v("and in the Frictionless Data forum:")]),e._v(" "),a("ul",[a("li",[a("a",{attrs:{href:"https://discuss.okfn.org/t/can-you-add-code-descriptions-to-a-data-package/",target:"_blank",rel:"noopener noreferrer"}},[e._v("https://discuss.okfn.org/t/can-you-add-code-descriptions-to-a-data-package/"),a("OutboundLink")],1)]),e._v(" "),a("li",[a("a",{attrs:{href:"https://discuss.okfn.org/t/something-like-rs-ordered-factors-or-enums-as-column-type/",target:"_blank",rel:"noopener noreferrer"}},[e._v("https://discuss.okfn.org/t/something-like-rs-ordered-factors-or-enums-as-column-type/"),a("OutboundLink")],1)])]),e._v(" "),a("p",[e._v("Finally, while we are unaware of any existing implementations intended for"),a("br"),e._v("\ngeneral use, it is likely that many users are already exploiting the fact that"),a("br"),e._v("\narbitrary fields may be added to the"),a("br"),e._v(" "),a("a",{attrs:{href:"https://specs.frictionlessdata.io//table-schema/",target:"_blank",rel:"noopener noreferrer"}},[e._v("table schema"),a("OutboundLink")],1),a("br"),e._v("\nto support internal implementations.")]),e._v(" "),a("h3",{attrs:{id:"proposed-extensions"}},[a("a",{staticClass:"header-anchor",attrs:{href:"#proposed-extensions"}},[e._v("#")]),e._v(" Proposed extensions")]),e._v(" "),a("p",[e._v("We propose two extensions to "),a("a",{attrs:{href:"https://specs.frictionlessdata.io/table-schema/",target:"_blank",rel:"noopener noreferrer"}},[e._v("Table Schema"),a("OutboundLink")],1),e._v(":")]),e._v(" "),a("ol",[a("li",[e._v("Add an optional field-specific "),a("code",[e._v("enumOrdered")]),e._v(" property, which can be used"),a("br"),e._v("\nwhen contructing a categorical (or factor) to indicate that the variable is"),a("br"),e._v("\nordinal.")]),e._v(" "),a("li",[e._v("Add an optional field-specific "),a("code",[e._v("enumLabels")]),e._v(" property for use when data are"),a("br"),e._v("\nstored using integer or other codes rather than using the category labels."),a("br"),e._v("\nThis contains an object mapping the codes appearing in the data (keys) to"),a("br"),e._v("\nwhat they mean (values), and can be used by software to construct"),a("br"),e._v("\ncorresponding value labels or categoricals (when supported) or to translate"),a("br"),e._v("\nthe values when reading the data.")])]),e._v(" "),a("p",[e._v("These extensions are fully backward compatible, since they are optional and"),a("br"),e._v("\nnot providing them is valid.")]),e._v(" "),a("p",[e._v("Here is an example of a categorical variable using extension (1):")]),e._v(" "),a("div",{staticClass:"language- extra-class"},[a("pre",{pre:!0,attrs:{class:"language-text"}},[a("code",[e._v('{\n "fields": [\n {\n "name": "physical_health",\n "type": "string",\n "constraints": {\n "enum": [\n "Poor",\n "Fair",\n "Good",\n "Very good",\n "Excellent",\n ]\n }\n "enumOrdered": true\n }\n ],\n "missingValues": ["Don\'t know","Refused","Not applicable"]\n}\n')])])]),a("p",[e._v("This is our preferred strategy, as it provides all of the information"),a("br"),e._v("\nnecessary to support the categorical functionality of the software packages"),a("br"),e._v("\nlisted above, while still yielding a useable result for software without such"),a("br"),e._v("\ncapability. As described below, value labels or categoricals can be created"),a("br"),e._v("\nautomatically based on the ordering of the values in the "),a("code",[e._v("enum")]),e._v(" array, and the"),a("br"),e._v(" "),a("code",[e._v("missingValues")]),e._v(" can be incorporated into the value labels or categoricals if"),a("br"),e._v("\ndesired. In those cases where it is desired to have more control over how the"),a("br"),e._v("\nvalue labels are constructed, this information can be stored in a separate"),a("br"),e._v("\nfile in JSON format or as part of a custom extension to the table schema."),a("br"),e._v("\nSince such instructions do not describe the data themselves (but only how a"),a("br"),e._v("\nspecific software package should handle them), and since they are often"),a("br"),e._v("\nsoftware- and/or user-specific, we argue that they should not be included in"),a("br"),e._v("\nthe official table schema.")]),e._v(" "),a("p",[e._v("Alternatively, those who wish to store their data in encoded form (e.g., this"),a("br"),e._v("\nis the default for data exports from "),a("a",{attrs:{href:"https://projectredcap.org",target:"_blank",rel:"noopener noreferrer"}},[e._v("REDCap"),a("OutboundLink")],1),e._v(", a"),a("br"),e._v("\ncommonly-used platform for collecting data for clinical studies) may use"),a("br"),e._v("\nextension (2) to do so:")]),e._v(" "),a("div",{staticClass:"language- extra-class"},[a("pre",{pre:!0,attrs:{class:"language-text"}},[a("code",[e._v('{\n "fields": [\n {\n "name": "physical_health",\n "type": "integer",\n "constraints": {\n "enum": [1,2,3,4,5]\n }\n "enumOrdered": true,\n "enumLabels": {\n "1": "Poor",\n "2": "Fair",\n "3": "Good",\n "4": "Very good",\n "5": "Excellent"\n }\n }\n ],\n "missingValues": ["Don\'t know","Refused","Not applicable"]\n}\n')])])]),a("p",[e._v("Note that although the field type is "),a("code",[e._v("integer")]),e._v(", the keys in the "),a("code",[e._v("enumLabels")]),a("br"),e._v("\nobject must be wrapped in double quotes because this is required by the JSON"),a("br"),e._v("\nfile format.")]),e._v(" "),a("p",[e._v("A second variant of the example above is the following:")]),e._v(" "),a("div",{staticClass:"language- extra-class"},[a("pre",{pre:!0,attrs:{class:"language-text"}},[a("code",[e._v('{\n "fields": [\n {\n "name": "physical_health",\n "type": "integer",\n "constraints": {\n "enum": [1,2,3,4,5]\n }\n "enumOrdered": true,\n "enumLabels": {\n "1": "Poor",\n "2": "Fair",\n "3": "Good",\n "4": "Very good",\n "5": "Excellent",\n ".a": "Don\'t know",\n ".b": "Refused",\n ".c": "Not applicable"\n }\n }\n ],\n "missingValues": [".a",".b",".c"]\n}\n')])])]),a("p",[e._v("This represents encoded data exported from software with support for value"),a("br"),e._v("\nlabels. The values "),a("code",[e._v(".a")]),e._v(", "),a("code",[e._v(".b")]),e._v(", etc. are known as "),a("em",[e._v("extended missing values")]),a("br"),e._v("\n(Stata and SAS only) and provide 26 unique missing values for numeric fields"),a("br"),e._v('\n(both integer and float) in addition to the system missing value ("'),a("code",[e._v(".")]),e._v('"); in'),a("br"),e._v("\nSPSS these would be replaced with specially designated integers, typically"),a("br"),e._v("\nnegative (e.g., -97, -98 and -99).")]),e._v(" "),a("h3",{attrs:{id:"specification-15"}},[a("a",{staticClass:"header-anchor",attrs:{href:"#specification-15"}},[e._v("#")]),e._v(" Specification")]),e._v(" "),a("ol",[a("li",[a("p",[e._v("A field with an "),a("code",[e._v("enum")]),e._v(" constraint or an "),a("code",[e._v("enumLabels")]),e._v(" property MAY have an"),a("br"),e._v(" "),a("code",[e._v("enumOrdered")]),e._v(" property that MUST be a boolean. A value of "),a("code",[e._v("true")]),e._v(" indicates"),a("br"),e._v("\nthat the field should be treated as having an ordinal scale of measurement,"),a("br"),e._v("\nwith the ordering given by the order of the field’s "),a("code",[e._v("enum")]),e._v(" array or by the"),a("br"),e._v("\nlexical order of the "),a("code",[e._v("enumLabels")]),e._v(" object’s keys, with the latter taking"),a("br"),e._v("\nprecedence. Fields without an "),a("code",[e._v("enum")]),e._v(" constraint or an "),a("code",[e._v("enumLabels")]),e._v(" property"),a("br"),e._v("\nor for which the "),a("code",[e._v("enumLabels")]),e._v(" keys do not include all values observed"),a("br"),e._v("\nin the data (excluding any values specified in the "),a("code",[e._v("missingValues")]),a("br"),e._v("\nproperty) MUST NOT have an "),a("code",[e._v("enumOrdered")]),e._v(" property since in that case the"),a("br"),e._v("\ncorrect ordering of the data is ambiguous. The absence of an "),a("code",[e._v("enumOrdered")]),a("br"),e._v("\nproperty MUST NOT be taken to imply "),a("code",[e._v("enumOrdered: false")]),e._v(".")])]),e._v(" "),a("li",[a("p",[e._v("A field MAY have an "),a("code",[e._v("enumLabels")]),e._v(" property that MUST be an object. This"),a("br"),e._v("\nproperty SHOULD be used to indicate how the values in the data (represented"),a("br"),e._v("\nby the object’s keys) are to be labeled or translated (represented by the"),a("br"),e._v("\ncorresponding value). As required by the JSON format, the object’s keys"),a("br"),e._v("\nmust be listed as strings (i.e., wrapped in double quotes). The keys MAY"),a("br"),e._v("\ninclude values that do not appear in the data and MAY omit some values that"),a("br"),e._v("\ndo appear in the data. For clarity and to avoid unintentional loss of"),a("br"),e._v("\ninformation, the object’s values SHOULD be unique.")])])]),e._v(" "),a("h3",{attrs:{id:"suggested-implementations"}},[a("a",{staticClass:"header-anchor",attrs:{href:"#suggested-implementations"}},[e._v("#")]),e._v(" Suggested implementations")]),e._v(" "),a("p",[e._v("Note: The use cases below address only "),a("em",[e._v("reading data")]),e._v(" from a Frictionless data"),a("br"),e._v("\npackage; it is assumed that implementations will also provide the ability to"),a("br"),e._v("\nwrite Frictionless data packages using the schema extensions proposed above."),a("br"),e._v("\nWe suggest two types of implementations:")]),e._v(" "),a("ol",[a("li",[a("p",[e._v("Additions to the official Python Frictionless Framework to generate"),a("br"),e._v("\nsoftware-specific scripts that may be executed by a specific software"),a("br"),e._v("\npackage to read data from a Frictionless data package and create the"),a("br"),e._v("\nappropriate value labels or categoricals, as described below. These"),a("br"),e._v("\nscripts can then be included along with the data in the package itself.")])]),e._v(" "),a("li",[a("p",[e._v("Software-specific extension packages that may be installed to permit users"),a("br"),e._v("\nof that software to read data from a Frictionless data package directly,"),a("br"),e._v("\nautomatically creating the appropriate value labels or categoricals as"),a("br"),e._v("\ndescribed below.")])])]),e._v(" "),a("p",[e._v("The advantage of (1) is that it doesn’t require users to install another"),a("br"),e._v("\nsoftware package, which may in some cases be difficult or impossible. The"),a("br"),e._v("\nadvantage of (2) is that it provides native support for working with"),a("br"),e._v("\nFrictionless data packages, and may be both easier and faster once the package"),a("br"),e._v("\nis installed. We are in the process of implementing both approaches for Stata;"),a("br"),e._v("\nimplementations for the other software listed above are straightforward.")]),e._v(" "),a("h4",{attrs:{id:"software-that-supports-value-labels-stata-sas-or-spss"}},[a("a",{staticClass:"header-anchor",attrs:{href:"#software-that-supports-value-labels-stata-sas-or-spss"}},[e._v("#")]),e._v(" Software that supports value labels (Stata, SAS or SPSS)")]),e._v(" "),a("ol",[a("li",[a("p",[e._v("In cases where a field has an "),a("code",[e._v("enum")]),e._v(" constraint but no "),a("code",[e._v("enumLabels")]),a("br"),e._v("\nproperty, automatically generate a value label mapping the integers 1, 2,"),a("br"),e._v("\n3, … to the "),a("code",[e._v("enum")]),e._v(" values in order, use this to encode the field (thereby"),a("br"),e._v("\nchanging its type from "),a("code",[e._v("string")]),e._v(" to "),a("code",[e._v("integer")]),e._v("), and attach the value label"),a("br"),e._v("\nto the field. Provide option to skip automatically dropping values"),a("br"),e._v("\nspecified in the "),a("code",[e._v("missingValues")]),e._v(" property and instead add them in order to"),a("br"),e._v("\nthe end of the value label, encoded using extended missing values if"),a("br"),e._v("\nsupported.")])]),e._v(" "),a("li",[a("p",[e._v("In cases where the data are stored in encoded form (e.g., as integers) and"),a("br"),e._v("\na corresponding "),a("code",[e._v("enumLabels")]),e._v(" property is present, and assuming that the"),a("br"),e._v("\nkeys in the "),a("code",[e._v("enumLabels")]),e._v(" object are limited to integers and extended"),a("br"),e._v("\nmissing values (if supported), use the "),a("code",[e._v("enumLabels")]),e._v(" object to generate a"),a("br"),e._v("\nvalue label and attach it to the field. As with (1), provide option to skip"),a("br"),e._v("\nautomatically dropping values specified in the "),a("code",[e._v("missingValues")]),e._v(" property and"),a("br"),e._v("\ninstead add them in order to the end of the value label, encoded using"),a("br"),e._v("\nextended missing values if supported.")])]),e._v(" "),a("li",[a("p",[e._v("Although none of Stata, SAS or SPSS currently permits designating a specific"),a("br"),e._v("\nvariable as ordered, Stata permits attaching arbitrary metadata to"),a("br"),e._v("\nindividual variables. Thus, in cases where the "),a("code",[e._v("enumOrdered")]),e._v(" property is"),a("br"),e._v("\npresent, this information can be stored in Stata to inform the analyst and"),a("br"),e._v("\nto prevent loss of information when generating Frictionless data packages"),a("br"),e._v("\nfrom within Stata.")])])]),e._v(" "),a("h4",{attrs:{id:"software-that-supports-categoricals-or-factors-pandas-r-julia"}},[a("a",{staticClass:"header-anchor",attrs:{href:"#software-that-supports-categoricals-or-factors-pandas-r-julia"}},[e._v("#")]),e._v(" Software that supports categoricals or factors (Pandas, R, Julia)")]),e._v(" "),a("ol",[a("li",[a("p",[e._v("In cases where a field has an "),a("code",[e._v("enum")]),e._v(" constraint but no "),a("code",[e._v("enumLabels")]),a("br"),e._v("\nproperty, automatically define a categorical or factor using the "),a("code",[e._v("enum")]),a("br"),e._v("\nvalues in order, and convert the variable to categorical or factor type"),a("br"),e._v("\nusing this definition. Provide option to skip automatically dropping values"),a("br"),e._v("\nspecified in the "),a("code",[e._v("missingValues")]),e._v(" property and instead add them in order to"),a("br"),e._v("\nthe end of the "),a("code",[e._v("enum")]),e._v(" values when defining the categorical or factor.")])]),e._v(" "),a("li",[a("p",[e._v("In cases where the data are stored in encoded form (e.g., as integers) and"),a("br"),e._v("\na corresponding "),a("code",[e._v("enumLabels")]),e._v(" property is present, translate the data using"),a("br"),e._v("\nthe "),a("code",[e._v("enumLabels")]),e._v(" object, define a categorical or factor using the values of"),a("br"),e._v("\nthe "),a("code",[e._v("enumLabels")]),e._v(" object in lexical order of the keys, and convert the"),a("br"),e._v("\nvariable to categorical or factor type using this definition. Provide"),a("br"),e._v("\noption to skip automatically dropping values specified in the"),a("br"),e._v(" "),a("code",[e._v("missingValues")]),e._v(" property and instead add them to the end of the"),a("br"),e._v(" "),a("code",[e._v("enumLabels")]),e._v(" values when defining the categorical or factor.")])]),e._v(" "),a("li",[a("p",[e._v("In cases where a field has an "),a("code",[e._v("enumOrdered")]),e._v(" property, use that when"),a("br"),e._v("\ndefining the categorical or factor.")])])]),e._v(" "),a("h4",{attrs:{id:"all-software"}},[a("a",{staticClass:"header-anchor",attrs:{href:"#all-software"}},[e._v("#")]),e._v(" All software")]),e._v(" "),a("p",[e._v("Although the extensions proposed here are intended primarily to support the"),a("br"),e._v("\nuse of value labels and categoricals in software that supports them, they also"),a("br"),e._v("\nprovide additional functionality when reading data into any software that can"),a("br"),e._v("\nhandle tabular data. Specifically, the "),a("code",[e._v("enumLabels")]),e._v(" property may be used to"),a("br"),e._v("\nsupport the use of enums even in cases where value labels or categoricals are"),a("br"),e._v("\nnot being used. For example, it is standard practice in software for analyzing"),a("br"),e._v("\ngenetic data to code sex as 0, 1 and 2 (corresponding to “Unknown”, “Male” and"),a("br"),e._v("\n“Female”) and affection status as 0, 1 and 2 (corresponding to “Unknown”,"),a("br"),e._v("\n“Unaffected” and “Affected”). In such cases, the "),a("code",[e._v("enumLabels")]),e._v(" property may be"),a("br"),e._v("\nused to confirm that the data follow the standard convention or to indicate"),a("br"),e._v("\nthat they deviate from it; it may also be used to translate those codes into"),a("br"),e._v("\nhuman-readable values, if desired.")]),e._v(" "),a("h3",{attrs:{id:"notes"}},[a("a",{staticClass:"header-anchor",attrs:{href:"#notes"}},[e._v("#")]),e._v(" Notes")]),e._v(" "),a("p",[e._v("While this pattern is designed as an extension to "),a("a",{attrs:{href:"https://specs.frictionlessdata.io/table-schema/",target:"_blank",rel:"noopener noreferrer"}},[e._v("Table Schema"),a("OutboundLink")],1),e._v(" fields, it could also be used to document "),a("code",[e._v("enum")]),e._v(" values of properties in "),a("a",{attrs:{href:"https://specs.frictionlessdata.io/profiles/",target:"_blank",rel:"noopener noreferrer"}},[e._v("profiles"),a("OutboundLink")],1),e._v(", such as contributor roles.")]),e._v(" "),a("p",[e._v("This pattern originally included a proposal to add an optional field-specific"),a("br"),e._v(" "),a("code",[e._v("missingValues")]),e._v(" property similar to that described in the pattern"),a("br"),e._v("\n“"),a("a",{attrs:{href:"https://specs.frictionlessdata.io/patterns/#missing-values-per-field",target:"_blank",rel:"noopener noreferrer"}},[e._v("missing values per field"),a("OutboundLink")],1),e._v("”"),a("br"),e._v("\nappearing in this document above. The objective was to provide a mechnanism to"),a("br"),e._v("\ndistinguish between so-called "),a("em",[e._v("system missing values")]),e._v(" (i.e., values that"),a("br"),e._v("\nindicate only that the corresponding data are missing) and other values that"),a("br"),e._v("\nconvey meaning but are typically excluded when fitting statistical models. The"),a("br"),e._v("\nlatter may be represented by "),a("em",[e._v("extended missing values")]),e._v(" ("),a("code",[e._v(".a")]),e._v(", "),a("code",[e._v(".b")]),e._v(", "),a("code",[e._v(".c")]),e._v(","),a("br"),e._v("\netc.) in Stata and SAS, or in SPSS by negative integers that are then"),a("br"),e._v("\ndesignated as missing by using the "),a("code",[e._v("MISSING VALUES")]),e._v(" command. For example,"),a("br"),e._v("\nvalues such as “NA”, “Not applicable”, “.”, etc. could be specified in the"),a("br"),e._v("\nresource level "),a("code",[e._v("missingValues")]),e._v(" property, while values such as “Don’t know” and"),a("br"),e._v("\n“Refused”—often used when generating tabular summaries and occasionally used"),a("br"),e._v("\nwhen fitting certain statistical models—could be specified in the"),a("br"),e._v("\ncorresponding field level "),a("code",[e._v("missingValues")]),e._v(" property. The former would still be"),a("br"),e._v("\nconverted to "),a("code",[e._v("null")]),e._v(" before type-specific string conversion (just as they are"),a("br"),e._v("\nnow), while the latter could be used by capable software when creating value"),a("br"),e._v("\nlabels or categoricals.")]),e._v(" "),a("p",[e._v("While this proposal was consistent with the principles outlined at the"),a("br"),e._v("\nbeginning (in particular, existing software would still yield a usable result"),a("br"),e._v("\nwhen reading the data), we realized that it would conflict with what appears"),a("br"),e._v("\nto be an emerging consensus regarding field-specific "),a("code",[e._v("missingValues")]),e._v("; i.e.,"),a("br"),e._v("\nthat they should "),a("em",[e._v("replace")]),e._v(" the less specific resource level "),a("code",[e._v("missingValues")]),a("br"),e._v("\nfor the corresponding field rather than be combined with them (see the discussion"),a("br"),e._v(" "),a("a",{attrs:{href:"https://github.com/frictionlessdata/specs/issues/551",target:"_blank",rel:"noopener noreferrer"}},[e._v("here"),a("OutboundLink")],1),e._v(" as well as the"),a("br"),e._v("\n“"),a("a",{attrs:{href:"https://specs.frictionlessdata.io/patterns/#missing-values-per-field",target:"_blank",rel:"noopener noreferrer"}},[e._v("missing values per field"),a("OutboundLink")],1),e._v("”"),a("br"),e._v("\npattern above). While there is good reason for replacing rather than combining"),a("br"),e._v("\nhere (e.g., it is more explicit), it would unfortunately conflict with the"),a("br"),e._v("\nidea of using the field-specific "),a("code",[e._v("missingValues")]),e._v(" in conjunction with the"),a("br"),e._v("\nresource level "),a("code",[e._v("missingValues")]),e._v(" as just described; namely, if the"),a("br"),e._v("\nfield-specific property replaced the resource level property then the system"),a("br"),e._v("\nmissing values would no longer be converted to "),a("code",[e._v("null")]),e._v(", as desired.")]),e._v(" "),a("p",[e._v("For this reason, we have dropped the proposal to add a field-specific"),a("br"),e._v(" "),a("code",[e._v("missingValues")]),e._v(" property from this pattern, and assert that implementation of"),a("br"),e._v("\nthis pattern by software should assume that if a field-specific "),a("code",[e._v("missingValues")]),a("br"),e._v("\nproperty is added to the"),a("br"),e._v(" "),a("a",{attrs:{href:"https://specs.frictionlessdata.io//table-schema/",target:"_blank",rel:"noopener noreferrer"}},[e._v("table schema"),a("OutboundLink")],1),a("br"),e._v("\nit should, if present, replace the resource level "),a("code",[e._v("missingValues")]),e._v(" property for"),a("br"),e._v("\nthe corresponding field. We do not believe that this change represents a"),a("br"),e._v("\nsubstantial limitation when creating value labels or categoricals, since"),a("br"),e._v("\nsystem missing values can typically be easily distinguished from other missing"),a("br"),e._v("\nvalues when exported in CSV format (e.g., “.” in Stata or SAS, “NA” in R, or"),a("br"),e._v("\n“” in Pandas).")])])}),[],!1,null,null,null);t.default=n.exports}}]); \ No newline at end of file diff --git a/assets/js/25.20ef3828.js b/assets/js/25.ef9b4db4.js similarity index 98% rename from assets/js/25.20ef3828.js rename to assets/js/25.ef9b4db4.js index 3e8a49bd..f073ac94 100644 --- a/assets/js/25.20ef3828.js +++ b/assets/js/25.ef9b4db4.js @@ -1 +1 @@ -(window.webpackJsonp=window.webpackJsonp||[]).push([[25],{394:function(a,e,t){"use strict";t.r(e);var r=t(26),o=Object(r.a)({},(function(){var a=this,e=a.$createElement,t=a._self._c||e;return t("ContentSlotsDistributor",{attrs:{"slot-key":a.$parent.slotKey}},[t("h1",{attrs:{id:"page-frontmatter-title"}},[t("a",{staticClass:"header-anchor",attrs:{href:"#page-frontmatter-title"}},[a._v("#")]),a._v(" "+a._s(a.$page.frontmatter.title))]),a._v(" "),t("p",[a._v(a._s(a.$page.frontmatter.abstract))]),a._v(" "),t("MetadataTable"),a._v(" "),t("h2",{attrs:{id:"language"}},[t("a",{staticClass:"header-anchor",attrs:{href:"#language"}},[a._v("#")]),a._v(" Language")]),a._v(" "),t("Language"),a._v(" "),t("h2",{attrs:{id:"introduction"}},[t("a",{staticClass:"header-anchor",attrs:{href:"#introduction"}},[a._v("#")]),a._v(" Introduction")]),a._v(" "),t("p",[a._v("Different kinds of data need different data and metadata formats. To support these different data and metadata formats we need to extend and specialise the generic Data Package. These specialized types of Data Package (or Data Resource) are termed "),t("strong",[a._v("profiles")]),a._v(". For example, there is a "),t("RouterLink",{attrs:{to:"/tabular-data-package/"}},[a._v("Tabular Data Package")]),a._v(" profile that specializes Data Packages specifically for tabular data.")],1),a._v(" "),t("p",[a._v("Thus, every Package and Resource descriptor has a profile. The namespace for the profile is the type of descriptor, so the default profile, if none is declared, is "),t("code",[a._v("data-package")]),a._v(" for a Package descriptor and "),t("code",[a._v("data-resource")]),a._v(" for a Resource descriptor.")]),a._v(" "),t("p",[a._v("In summary, an extension of Data Package may be formalised as a profile. A profile is a Data Package which extends the default specification towards more specific needs.")]),a._v(" "),t("h3",{attrs:{id:"profile-property"}},[t("a",{staticClass:"header-anchor",attrs:{href:"#profile-property"}},[a._v("#")]),a._v(" "),t("code",[a._v("profile")]),a._v(" Property")]),a._v(" "),t("p",[a._v("In addition to the concept, we need an explicit way for publishers to state the profile they are using and consumers to discover this.")]),a._v(" "),t("p",[a._v("Thus, we have a "),t("code",[a._v("profile")]),a._v(" property that declares the profile for the descriptor for this Package or Resource. For the default Data Package and Data Resource descriptor, this SHOULD be present with a value of "),t("code",[a._v("data-package")]),a._v("/"),t("code",[a._v("data-resource")]),a._v(", but if not, the absence of a profile is equivalent to setting "),t("code",[a._v('"profile": "data-package"')]),a._v("/ "),t("code",[a._v('"profile": "data-resource"')]),a._v(".")]),a._v(" "),t("p",[a._v("Custom profiles MUST have a profile property, where the value is a unique identifier for that profile. This unique identifier "),t("code",[a._v("MUST")]),a._v(" be a string and can be in one of two forms. It can be an id from the official "),t("a",{attrs:{href:"/schemas/registry.json"}},[a._v("Data Package Schema Registry")]),a._v(", or, a fully-qualified URL that points directly to a JSON Schema that can be used to validate the profile.")]),a._v(" "),t("p",[a._v("As part of the Frictionless Data Specifications project, we publish a number of Data Package profiles such as:")]),a._v(" "),t("ul",[t("li",[t("RouterLink",{attrs:{to:"/tabular-data-package/"}},[a._v("Tabular Data Package")])],1),a._v(" "),t("li",[t("RouterLink",{attrs:{to:"/fiscal-data-package/"}},[a._v("Fiscal Data Package")])],1)]),a._v(" "),t("p",[a._v("We also publish the following Data Resource profiles:")]),a._v(" "),t("ul",[t("li",[t("RouterLink",{attrs:{to:"/tabular-data-resource/"}},[a._v("Tabular Data Resource")])],1)])],1)}),[],!1,null,null,null);e.default=o.exports}}]); \ No newline at end of file +(window.webpackJsonp=window.webpackJsonp||[]).push([[25],{395:function(a,e,t){"use strict";t.r(e);var r=t(26),o=Object(r.a)({},(function(){var a=this,e=a.$createElement,t=a._self._c||e;return t("ContentSlotsDistributor",{attrs:{"slot-key":a.$parent.slotKey}},[t("h1",{attrs:{id:"page-frontmatter-title"}},[t("a",{staticClass:"header-anchor",attrs:{href:"#page-frontmatter-title"}},[a._v("#")]),a._v(" "+a._s(a.$page.frontmatter.title))]),a._v(" "),t("p",[a._v(a._s(a.$page.frontmatter.abstract))]),a._v(" "),t("MetadataTable"),a._v(" "),t("h2",{attrs:{id:"language"}},[t("a",{staticClass:"header-anchor",attrs:{href:"#language"}},[a._v("#")]),a._v(" Language")]),a._v(" "),t("Language"),a._v(" "),t("h2",{attrs:{id:"introduction"}},[t("a",{staticClass:"header-anchor",attrs:{href:"#introduction"}},[a._v("#")]),a._v(" Introduction")]),a._v(" "),t("p",[a._v("Different kinds of data need different data and metadata formats. To support these different data and metadata formats we need to extend and specialise the generic Data Package. These specialized types of Data Package (or Data Resource) are termed "),t("strong",[a._v("profiles")]),a._v(". For example, there is a "),t("RouterLink",{attrs:{to:"/tabular-data-package/"}},[a._v("Tabular Data Package")]),a._v(" profile that specializes Data Packages specifically for tabular data.")],1),a._v(" "),t("p",[a._v("Thus, every Package and Resource descriptor has a profile. The namespace for the profile is the type of descriptor, so the default profile, if none is declared, is "),t("code",[a._v("data-package")]),a._v(" for a Package descriptor and "),t("code",[a._v("data-resource")]),a._v(" for a Resource descriptor.")]),a._v(" "),t("p",[a._v("In summary, an extension of Data Package may be formalised as a profile. A profile is a Data Package which extends the default specification towards more specific needs.")]),a._v(" "),t("h3",{attrs:{id:"profile-property"}},[t("a",{staticClass:"header-anchor",attrs:{href:"#profile-property"}},[a._v("#")]),a._v(" "),t("code",[a._v("profile")]),a._v(" Property")]),a._v(" "),t("p",[a._v("In addition to the concept, we need an explicit way for publishers to state the profile they are using and consumers to discover this.")]),a._v(" "),t("p",[a._v("Thus, we have a "),t("code",[a._v("profile")]),a._v(" property that declares the profile for the descriptor for this Package or Resource. For the default Data Package and Data Resource descriptor, this SHOULD be present with a value of "),t("code",[a._v("data-package")]),a._v("/"),t("code",[a._v("data-resource")]),a._v(", but if not, the absence of a profile is equivalent to setting "),t("code",[a._v('"profile": "data-package"')]),a._v("/ "),t("code",[a._v('"profile": "data-resource"')]),a._v(".")]),a._v(" "),t("p",[a._v("Custom profiles MUST have a profile property, where the value is a unique identifier for that profile. This unique identifier "),t("code",[a._v("MUST")]),a._v(" be a string and can be in one of two forms. It can be an id from the official "),t("a",{attrs:{href:"/schemas/registry.json"}},[a._v("Data Package Schema Registry")]),a._v(", or, a fully-qualified URL that points directly to a JSON Schema that can be used to validate the profile.")]),a._v(" "),t("p",[a._v("As part of the Frictionless Data Specifications project, we publish a number of Data Package profiles such as:")]),a._v(" "),t("ul",[t("li",[t("RouterLink",{attrs:{to:"/tabular-data-package/"}},[a._v("Tabular Data Package")])],1),a._v(" "),t("li",[t("RouterLink",{attrs:{to:"/fiscal-data-package/"}},[a._v("Fiscal Data Package")])],1)]),a._v(" "),t("p",[a._v("We also publish the following Data Resource profiles:")]),a._v(" "),t("ul",[t("li",[t("RouterLink",{attrs:{to:"/tabular-data-resource/"}},[a._v("Tabular Data Resource")])],1)])],1)}),[],!1,null,null,null);e.default=o.exports}}]); \ No newline at end of file diff --git a/assets/js/28.ae08d307.js b/assets/js/28.ec459688.js similarity index 99% rename from assets/js/28.ae08d307.js rename to assets/js/28.ec459688.js index dea08d81..3f4a2895 100644 --- a/assets/js/28.ae08d307.js +++ b/assets/js/28.ec459688.js @@ -1 +1 @@ -(window.webpackJsonp=window.webpackJsonp||[]).push([[28],{400:function(a,t,s){"use strict";s.r(t);var e=s(26),r=Object(e.a)({},(function(){var a=this,t=a.$createElement,s=a._self._c||t;return s("ContentSlotsDistributor",{attrs:{"slot-key":a.$parent.slotKey}},[s("h1",{attrs:{id:"page-frontmatter-title"}},[s("a",{staticClass:"header-anchor",attrs:{href:"#page-frontmatter-title"}},[a._v("#")]),a._v(" "+a._s(a.$page.frontmatter.title))]),a._v(" "),s("p",[a._v(a._s(a.$page.frontmatter.abstract))]),a._v(" "),s("MetadataTable"),a._v(" "),s("h2",{attrs:{id:"language"}},[s("a",{staticClass:"header-anchor",attrs:{href:"#language"}},[a._v("#")]),a._v(" Language")]),a._v(" "),s("Language"),a._v(" "),s("h2",{attrs:{id:"introduction"}},[s("a",{staticClass:"header-anchor",attrs:{href:"#introduction"}},[a._v("#")]),a._v(" Introduction")]),a._v(" "),s("p",[a._v("Tabular Data Package is a simple container format used for publishing and sharing tabular-style data. The format’s focus is on simplicity and ease of use, especially online. In addition, the format is focused on data that can be presented in a tabular structure and in making it easy to produce (and consume) tabular data packages from spreadsheets and relational databases.")]),a._v(" "),s("p",[a._v("The key features of this format are the following:")]),a._v(" "),s("ul",[s("li",[a._v("CSV (comma separated variables) for data files")]),a._v(" "),s("li",[a._v("Single JSON file (datapackage.json) to describe the dataset including a schema for data files")]),a._v(" "),s("li",[a._v("Reuse of existing work including other Frictionless Data specifications")])]),a._v(" "),s("p",[a._v("As suggested by the name, Tabular Data Package extends and specializes the "),s("RouterLink",{attrs:{to:"/data-package/"}},[a._v("Data Package")]),a._v(" spec for the specific case where the data is tabular.")],1),a._v(" "),s("h3",{attrs:{id:"why-csv"}},[s("a",{staticClass:"header-anchor",attrs:{href:"#why-csv"}},[a._v("#")]),a._v(" Why CSV")]),a._v(" "),s("p",[a._v("We chose CSV as the data format for the Tabular Data Package specification because:")]),a._v(" "),s("ol",[s("li",[a._v("CSV is very simple – it is possibly "),s("em",[a._v("the")]),a._v(" most simple data format")]),a._v(" "),s("li",[a._v("CSV is tabular-oriented. Most data structures are either tabular or can be transformed to a tabular structure by some form of normalization")]),a._v(" "),s("li",[a._v("It is open and the “standard” is well-known")]),a._v(" "),s("li",[a._v("It is widely supported - practically every spreadsheet program, relational database and programming language in existence can handle CSV in some form or other")]),a._v(" "),s("li",[a._v("It is text-based and therefore amenable to manipulation and access from a wide range of standard tools (including revision control systems such as git, mercurial and subversion)")]),a._v(" "),s("li",[a._v("It is line-oriented which means it can be incrementally processed - you do not need to read an entire file to extract a single row. For similar reasons it means that the format supports streaming.")])]),a._v(" "),s("p",[a._v("More informally:")]),a._v(" "),s("blockquote",[s("p",[a._v("CSV is the data Kalashnikov: not pretty, but many wars have been"),s("br"),a._v("\nfought with it and kids can use it."),s("br"),a._v("\n["),s("a",{attrs:{href:"https://twitter.com/pudo/status/248473299741446144",target:"_blank",rel:"noopener noreferrer"}},[a._v("@pudo"),s("OutboundLink")],1),a._v(" (Friedrich"),s("br"),a._v("\nLindenberg)]")])]),a._v(" "),s("blockquote",[s("p",[a._v("CSV is the ultimate simple, standard data format - streamable,"),s("br"),a._v("\ntext-based, no need for proprietary tools etc [@rufuspollock (Rufus"),s("br"),a._v("\nPollock)]")])]),a._v(" "),s("h2",{attrs:{id:"specification"}},[s("a",{staticClass:"header-anchor",attrs:{href:"#specification"}},[a._v("#")]),a._v(" Specification")]),a._v(" "),s("p",[a._v("Tabular Data Package builds directly on the "),s("RouterLink",{attrs:{to:"/data-package/"}},[a._v("Data Package")]),a._v(" specification. Thus a Tabular Data Package "),s("code",[a._v("MUST")]),a._v(" be a Data Package and conform to the "),s("RouterLink",{attrs:{to:"/data-package/"}},[a._v("Data Package specification")]),a._v(".")],1),a._v(" "),s("p",[a._v("Tabular Data Package has the following requirements over and above those imposed by "),s("RouterLink",{attrs:{to:"/data-package/"}},[a._v("Data Package")]),a._v(":")],1),a._v(" "),s("ul",[s("li",[a._v("There "),s("code",[a._v("MUST")]),a._v(" be at least one "),s("code",[a._v("resource")]),a._v(" in the "),s("code",[a._v("resources")]),a._v(" "),s("code",[a._v("array")])]),a._v(" "),s("li",[a._v("There "),s("code",[a._v("MUST")]),a._v(" be a "),s("code",[a._v("profile")]),a._v(" property with the value "),s("code",[a._v("tabular-data-package")])]),a._v(" "),s("li",[a._v("Each "),s("code",[a._v("resource")]),a._v(" "),s("code",[a._v("MUST")]),a._v(" be a "),s("RouterLink",{attrs:{to:"/tabular-data-resource/"}},[a._v("Tabular Data Resource")])],1)]),a._v(" "),s("h3",{attrs:{id:"example"}},[s("a",{staticClass:"header-anchor",attrs:{href:"#example"}},[a._v("#")]),a._v(" Example")]),a._v(" "),s("p",[a._v("Here’s an example of a minimal tabular data package:")]),a._v(" "),s("p",[a._v("On disk we have 2 files:")]),a._v(" "),s("div",{staticClass:"language- extra-class"},[s("pre",{pre:!0,attrs:{class:"language-text"}},[s("code",[a._v("data.csv\ndatapackage.json\n")])])]),s("p",[s("code",[a._v("data.csv")]),a._v(" looks like:")]),a._v(" "),s("div",{staticClass:"language- extra-class"},[s("pre",{pre:!0,attrs:{class:"language-text"}},[s("code",[a._v("var1,var2,var3\nA,1,2.1\nB,3,4.5\n")])])]),s("p",[s("code",[a._v("datapackage.json")]),a._v(" looks like:")]),a._v(" "),s("div",{staticClass:"language-javascript extra-class"},[s("pre",{pre:!0,attrs:{class:"language-javascript"}},[s("code",[s("span",{pre:!0,attrs:{class:"token punctuation"}},[a._v("{")]),a._v("\n "),s("span",{pre:!0,attrs:{class:"token string"}},[a._v('"profile"')]),s("span",{pre:!0,attrs:{class:"token operator"}},[a._v(":")]),a._v(" "),s("span",{pre:!0,attrs:{class:"token string"}},[a._v('"tabular-data-package"')]),s("span",{pre:!0,attrs:{class:"token punctuation"}},[a._v(",")]),a._v("\n "),s("span",{pre:!0,attrs:{class:"token string"}},[a._v('"name"')]),s("span",{pre:!0,attrs:{class:"token operator"}},[a._v(":")]),a._v(" "),s("span",{pre:!0,attrs:{class:"token string"}},[a._v('"my-dataset"')]),s("span",{pre:!0,attrs:{class:"token punctuation"}},[a._v(",")]),a._v("\n "),s("span",{pre:!0,attrs:{class:"token comment"}},[a._v("// here we list the data files in this dataset")]),a._v("\n "),s("span",{pre:!0,attrs:{class:"token string"}},[a._v('"resources"')]),s("span",{pre:!0,attrs:{class:"token operator"}},[a._v(":")]),a._v(" "),s("span",{pre:!0,attrs:{class:"token punctuation"}},[a._v("[")]),a._v("\n "),s("span",{pre:!0,attrs:{class:"token punctuation"}},[a._v("{")]),a._v("\n "),s("span",{pre:!0,attrs:{class:"token string"}},[a._v('"profile"')]),s("span",{pre:!0,attrs:{class:"token operator"}},[a._v(":")]),a._v(" "),s("span",{pre:!0,attrs:{class:"token string"}},[a._v('"tabular-data-resource"')]),s("span",{pre:!0,attrs:{class:"token punctuation"}},[a._v(",")]),a._v("\n "),s("span",{pre:!0,attrs:{class:"token string"}},[a._v('"name"')]),s("span",{pre:!0,attrs:{class:"token operator"}},[a._v(":")]),a._v(" "),s("span",{pre:!0,attrs:{class:"token string"}},[a._v('"data"')]),s("span",{pre:!0,attrs:{class:"token punctuation"}},[a._v(",")]),a._v("\n "),s("span",{pre:!0,attrs:{class:"token string"}},[a._v('"path"')]),s("span",{pre:!0,attrs:{class:"token operator"}},[a._v(":")]),a._v(" "),s("span",{pre:!0,attrs:{class:"token string"}},[a._v('"data.csv"')]),s("span",{pre:!0,attrs:{class:"token punctuation"}},[a._v(",")]),a._v("\n "),s("span",{pre:!0,attrs:{class:"token string"}},[a._v('"schema"')]),s("span",{pre:!0,attrs:{class:"token operator"}},[a._v(":")]),a._v(" "),s("span",{pre:!0,attrs:{class:"token punctuation"}},[a._v("{")]),a._v("\n "),s("span",{pre:!0,attrs:{class:"token string"}},[a._v('"fields"')]),s("span",{pre:!0,attrs:{class:"token operator"}},[a._v(":")]),a._v(" "),s("span",{pre:!0,attrs:{class:"token punctuation"}},[a._v("[")]),a._v("\n "),s("span",{pre:!0,attrs:{class:"token punctuation"}},[a._v("{")]),a._v("\n "),s("span",{pre:!0,attrs:{class:"token string"}},[a._v('"name"')]),s("span",{pre:!0,attrs:{class:"token operator"}},[a._v(":")]),a._v(" "),s("span",{pre:!0,attrs:{class:"token string"}},[a._v('"var1"')]),s("span",{pre:!0,attrs:{class:"token punctuation"}},[a._v(",")]),a._v("\n "),s("span",{pre:!0,attrs:{class:"token string"}},[a._v('"type"')]),s("span",{pre:!0,attrs:{class:"token operator"}},[a._v(":")]),a._v(" "),s("span",{pre:!0,attrs:{class:"token string"}},[a._v('"string"')]),a._v("\n "),s("span",{pre:!0,attrs:{class:"token punctuation"}},[a._v("}")]),s("span",{pre:!0,attrs:{class:"token punctuation"}},[a._v(",")]),a._v("\n "),s("span",{pre:!0,attrs:{class:"token punctuation"}},[a._v("{")]),a._v("\n "),s("span",{pre:!0,attrs:{class:"token string"}},[a._v('"name"')]),s("span",{pre:!0,attrs:{class:"token operator"}},[a._v(":")]),a._v(" "),s("span",{pre:!0,attrs:{class:"token string"}},[a._v('"var2"')]),s("span",{pre:!0,attrs:{class:"token punctuation"}},[a._v(",")]),a._v("\n "),s("span",{pre:!0,attrs:{class:"token string"}},[a._v('"type"')]),s("span",{pre:!0,attrs:{class:"token operator"}},[a._v(":")]),a._v(" "),s("span",{pre:!0,attrs:{class:"token string"}},[a._v('"integer"')]),a._v("\n "),s("span",{pre:!0,attrs:{class:"token punctuation"}},[a._v("}")]),s("span",{pre:!0,attrs:{class:"token punctuation"}},[a._v(",")]),a._v("\n "),s("span",{pre:!0,attrs:{class:"token punctuation"}},[a._v("{")]),a._v("\n "),s("span",{pre:!0,attrs:{class:"token string"}},[a._v('"name"')]),s("span",{pre:!0,attrs:{class:"token operator"}},[a._v(":")]),a._v(" "),s("span",{pre:!0,attrs:{class:"token string"}},[a._v('"var3"')]),s("span",{pre:!0,attrs:{class:"token punctuation"}},[a._v(",")]),a._v("\n "),s("span",{pre:!0,attrs:{class:"token string"}},[a._v('"type"')]),s("span",{pre:!0,attrs:{class:"token operator"}},[a._v(":")]),a._v(" "),s("span",{pre:!0,attrs:{class:"token string"}},[a._v('"number"')]),a._v("\n "),s("span",{pre:!0,attrs:{class:"token punctuation"}},[a._v("}")]),a._v("\n "),s("span",{pre:!0,attrs:{class:"token punctuation"}},[a._v("]")]),a._v("\n "),s("span",{pre:!0,attrs:{class:"token punctuation"}},[a._v("}")]),a._v("\n "),s("span",{pre:!0,attrs:{class:"token punctuation"}},[a._v("}")]),a._v("\n "),s("span",{pre:!0,attrs:{class:"token punctuation"}},[a._v("]")]),a._v("\n"),s("span",{pre:!0,attrs:{class:"token punctuation"}},[a._v("}")]),a._v("\n\n")])])])],1)}),[],!1,null,null,null);t.default=r.exports}}]); \ No newline at end of file +(window.webpackJsonp=window.webpackJsonp||[]).push([[28],{399:function(a,t,s){"use strict";s.r(t);var e=s(26),r=Object(e.a)({},(function(){var a=this,t=a.$createElement,s=a._self._c||t;return s("ContentSlotsDistributor",{attrs:{"slot-key":a.$parent.slotKey}},[s("h1",{attrs:{id:"page-frontmatter-title"}},[s("a",{staticClass:"header-anchor",attrs:{href:"#page-frontmatter-title"}},[a._v("#")]),a._v(" "+a._s(a.$page.frontmatter.title))]),a._v(" "),s("p",[a._v(a._s(a.$page.frontmatter.abstract))]),a._v(" "),s("MetadataTable"),a._v(" "),s("h2",{attrs:{id:"language"}},[s("a",{staticClass:"header-anchor",attrs:{href:"#language"}},[a._v("#")]),a._v(" Language")]),a._v(" "),s("Language"),a._v(" "),s("h2",{attrs:{id:"introduction"}},[s("a",{staticClass:"header-anchor",attrs:{href:"#introduction"}},[a._v("#")]),a._v(" Introduction")]),a._v(" "),s("p",[a._v("Tabular Data Package is a simple container format used for publishing and sharing tabular-style data. The format’s focus is on simplicity and ease of use, especially online. In addition, the format is focused on data that can be presented in a tabular structure and in making it easy to produce (and consume) tabular data packages from spreadsheets and relational databases.")]),a._v(" "),s("p",[a._v("The key features of this format are the following:")]),a._v(" "),s("ul",[s("li",[a._v("CSV (comma separated variables) for data files")]),a._v(" "),s("li",[a._v("Single JSON file (datapackage.json) to describe the dataset including a schema for data files")]),a._v(" "),s("li",[a._v("Reuse of existing work including other Frictionless Data specifications")])]),a._v(" "),s("p",[a._v("As suggested by the name, Tabular Data Package extends and specializes the "),s("RouterLink",{attrs:{to:"/data-package/"}},[a._v("Data Package")]),a._v(" spec for the specific case where the data is tabular.")],1),a._v(" "),s("h3",{attrs:{id:"why-csv"}},[s("a",{staticClass:"header-anchor",attrs:{href:"#why-csv"}},[a._v("#")]),a._v(" Why CSV")]),a._v(" "),s("p",[a._v("We chose CSV as the data format for the Tabular Data Package specification because:")]),a._v(" "),s("ol",[s("li",[a._v("CSV is very simple – it is possibly "),s("em",[a._v("the")]),a._v(" most simple data format")]),a._v(" "),s("li",[a._v("CSV is tabular-oriented. Most data structures are either tabular or can be transformed to a tabular structure by some form of normalization")]),a._v(" "),s("li",[a._v("It is open and the “standard” is well-known")]),a._v(" "),s("li",[a._v("It is widely supported - practically every spreadsheet program, relational database and programming language in existence can handle CSV in some form or other")]),a._v(" "),s("li",[a._v("It is text-based and therefore amenable to manipulation and access from a wide range of standard tools (including revision control systems such as git, mercurial and subversion)")]),a._v(" "),s("li",[a._v("It is line-oriented which means it can be incrementally processed - you do not need to read an entire file to extract a single row. For similar reasons it means that the format supports streaming.")])]),a._v(" "),s("p",[a._v("More informally:")]),a._v(" "),s("blockquote",[s("p",[a._v("CSV is the data Kalashnikov: not pretty, but many wars have been"),s("br"),a._v("\nfought with it and kids can use it."),s("br"),a._v("\n["),s("a",{attrs:{href:"https://twitter.com/pudo/status/248473299741446144",target:"_blank",rel:"noopener noreferrer"}},[a._v("@pudo"),s("OutboundLink")],1),a._v(" (Friedrich"),s("br"),a._v("\nLindenberg)]")])]),a._v(" "),s("blockquote",[s("p",[a._v("CSV is the ultimate simple, standard data format - streamable,"),s("br"),a._v("\ntext-based, no need for proprietary tools etc [@rufuspollock (Rufus"),s("br"),a._v("\nPollock)]")])]),a._v(" "),s("h2",{attrs:{id:"specification"}},[s("a",{staticClass:"header-anchor",attrs:{href:"#specification"}},[a._v("#")]),a._v(" Specification")]),a._v(" "),s("p",[a._v("Tabular Data Package builds directly on the "),s("RouterLink",{attrs:{to:"/data-package/"}},[a._v("Data Package")]),a._v(" specification. Thus a Tabular Data Package "),s("code",[a._v("MUST")]),a._v(" be a Data Package and conform to the "),s("RouterLink",{attrs:{to:"/data-package/"}},[a._v("Data Package specification")]),a._v(".")],1),a._v(" "),s("p",[a._v("Tabular Data Package has the following requirements over and above those imposed by "),s("RouterLink",{attrs:{to:"/data-package/"}},[a._v("Data Package")]),a._v(":")],1),a._v(" "),s("ul",[s("li",[a._v("There "),s("code",[a._v("MUST")]),a._v(" be at least one "),s("code",[a._v("resource")]),a._v(" in the "),s("code",[a._v("resources")]),a._v(" "),s("code",[a._v("array")])]),a._v(" "),s("li",[a._v("There "),s("code",[a._v("MUST")]),a._v(" be a "),s("code",[a._v("profile")]),a._v(" property with the value "),s("code",[a._v("tabular-data-package")])]),a._v(" "),s("li",[a._v("Each "),s("code",[a._v("resource")]),a._v(" "),s("code",[a._v("MUST")]),a._v(" be a "),s("RouterLink",{attrs:{to:"/tabular-data-resource/"}},[a._v("Tabular Data Resource")])],1)]),a._v(" "),s("h3",{attrs:{id:"example"}},[s("a",{staticClass:"header-anchor",attrs:{href:"#example"}},[a._v("#")]),a._v(" Example")]),a._v(" "),s("p",[a._v("Here’s an example of a minimal tabular data package:")]),a._v(" "),s("p",[a._v("On disk we have 2 files:")]),a._v(" "),s("div",{staticClass:"language- extra-class"},[s("pre",{pre:!0,attrs:{class:"language-text"}},[s("code",[a._v("data.csv\ndatapackage.json\n")])])]),s("p",[s("code",[a._v("data.csv")]),a._v(" looks like:")]),a._v(" "),s("div",{staticClass:"language- extra-class"},[s("pre",{pre:!0,attrs:{class:"language-text"}},[s("code",[a._v("var1,var2,var3\nA,1,2.1\nB,3,4.5\n")])])]),s("p",[s("code",[a._v("datapackage.json")]),a._v(" looks like:")]),a._v(" "),s("div",{staticClass:"language-javascript extra-class"},[s("pre",{pre:!0,attrs:{class:"language-javascript"}},[s("code",[s("span",{pre:!0,attrs:{class:"token punctuation"}},[a._v("{")]),a._v("\n "),s("span",{pre:!0,attrs:{class:"token string"}},[a._v('"profile"')]),s("span",{pre:!0,attrs:{class:"token operator"}},[a._v(":")]),a._v(" "),s("span",{pre:!0,attrs:{class:"token string"}},[a._v('"tabular-data-package"')]),s("span",{pre:!0,attrs:{class:"token punctuation"}},[a._v(",")]),a._v("\n "),s("span",{pre:!0,attrs:{class:"token string"}},[a._v('"name"')]),s("span",{pre:!0,attrs:{class:"token operator"}},[a._v(":")]),a._v(" "),s("span",{pre:!0,attrs:{class:"token string"}},[a._v('"my-dataset"')]),s("span",{pre:!0,attrs:{class:"token punctuation"}},[a._v(",")]),a._v("\n "),s("span",{pre:!0,attrs:{class:"token comment"}},[a._v("// here we list the data files in this dataset")]),a._v("\n "),s("span",{pre:!0,attrs:{class:"token string"}},[a._v('"resources"')]),s("span",{pre:!0,attrs:{class:"token operator"}},[a._v(":")]),a._v(" "),s("span",{pre:!0,attrs:{class:"token punctuation"}},[a._v("[")]),a._v("\n "),s("span",{pre:!0,attrs:{class:"token punctuation"}},[a._v("{")]),a._v("\n "),s("span",{pre:!0,attrs:{class:"token string"}},[a._v('"profile"')]),s("span",{pre:!0,attrs:{class:"token operator"}},[a._v(":")]),a._v(" "),s("span",{pre:!0,attrs:{class:"token string"}},[a._v('"tabular-data-resource"')]),s("span",{pre:!0,attrs:{class:"token punctuation"}},[a._v(",")]),a._v("\n "),s("span",{pre:!0,attrs:{class:"token string"}},[a._v('"name"')]),s("span",{pre:!0,attrs:{class:"token operator"}},[a._v(":")]),a._v(" "),s("span",{pre:!0,attrs:{class:"token string"}},[a._v('"data"')]),s("span",{pre:!0,attrs:{class:"token punctuation"}},[a._v(",")]),a._v("\n "),s("span",{pre:!0,attrs:{class:"token string"}},[a._v('"path"')]),s("span",{pre:!0,attrs:{class:"token operator"}},[a._v(":")]),a._v(" "),s("span",{pre:!0,attrs:{class:"token string"}},[a._v('"data.csv"')]),s("span",{pre:!0,attrs:{class:"token punctuation"}},[a._v(",")]),a._v("\n "),s("span",{pre:!0,attrs:{class:"token string"}},[a._v('"schema"')]),s("span",{pre:!0,attrs:{class:"token operator"}},[a._v(":")]),a._v(" "),s("span",{pre:!0,attrs:{class:"token punctuation"}},[a._v("{")]),a._v("\n "),s("span",{pre:!0,attrs:{class:"token string"}},[a._v('"fields"')]),s("span",{pre:!0,attrs:{class:"token operator"}},[a._v(":")]),a._v(" "),s("span",{pre:!0,attrs:{class:"token punctuation"}},[a._v("[")]),a._v("\n "),s("span",{pre:!0,attrs:{class:"token punctuation"}},[a._v("{")]),a._v("\n "),s("span",{pre:!0,attrs:{class:"token string"}},[a._v('"name"')]),s("span",{pre:!0,attrs:{class:"token operator"}},[a._v(":")]),a._v(" "),s("span",{pre:!0,attrs:{class:"token string"}},[a._v('"var1"')]),s("span",{pre:!0,attrs:{class:"token punctuation"}},[a._v(",")]),a._v("\n "),s("span",{pre:!0,attrs:{class:"token string"}},[a._v('"type"')]),s("span",{pre:!0,attrs:{class:"token operator"}},[a._v(":")]),a._v(" "),s("span",{pre:!0,attrs:{class:"token string"}},[a._v('"string"')]),a._v("\n "),s("span",{pre:!0,attrs:{class:"token punctuation"}},[a._v("}")]),s("span",{pre:!0,attrs:{class:"token punctuation"}},[a._v(",")]),a._v("\n "),s("span",{pre:!0,attrs:{class:"token punctuation"}},[a._v("{")]),a._v("\n "),s("span",{pre:!0,attrs:{class:"token string"}},[a._v('"name"')]),s("span",{pre:!0,attrs:{class:"token operator"}},[a._v(":")]),a._v(" "),s("span",{pre:!0,attrs:{class:"token string"}},[a._v('"var2"')]),s("span",{pre:!0,attrs:{class:"token punctuation"}},[a._v(",")]),a._v("\n "),s("span",{pre:!0,attrs:{class:"token string"}},[a._v('"type"')]),s("span",{pre:!0,attrs:{class:"token operator"}},[a._v(":")]),a._v(" "),s("span",{pre:!0,attrs:{class:"token string"}},[a._v('"integer"')]),a._v("\n "),s("span",{pre:!0,attrs:{class:"token punctuation"}},[a._v("}")]),s("span",{pre:!0,attrs:{class:"token punctuation"}},[a._v(",")]),a._v("\n "),s("span",{pre:!0,attrs:{class:"token punctuation"}},[a._v("{")]),a._v("\n "),s("span",{pre:!0,attrs:{class:"token string"}},[a._v('"name"')]),s("span",{pre:!0,attrs:{class:"token operator"}},[a._v(":")]),a._v(" "),s("span",{pre:!0,attrs:{class:"token string"}},[a._v('"var3"')]),s("span",{pre:!0,attrs:{class:"token punctuation"}},[a._v(",")]),a._v("\n "),s("span",{pre:!0,attrs:{class:"token string"}},[a._v('"type"')]),s("span",{pre:!0,attrs:{class:"token operator"}},[a._v(":")]),a._v(" "),s("span",{pre:!0,attrs:{class:"token string"}},[a._v('"number"')]),a._v("\n "),s("span",{pre:!0,attrs:{class:"token punctuation"}},[a._v("}")]),a._v("\n "),s("span",{pre:!0,attrs:{class:"token punctuation"}},[a._v("]")]),a._v("\n "),s("span",{pre:!0,attrs:{class:"token punctuation"}},[a._v("}")]),a._v("\n "),s("span",{pre:!0,attrs:{class:"token punctuation"}},[a._v("}")]),a._v("\n "),s("span",{pre:!0,attrs:{class:"token punctuation"}},[a._v("]")]),a._v("\n"),s("span",{pre:!0,attrs:{class:"token punctuation"}},[a._v("}")]),a._v("\n\n")])])])],1)}),[],!1,null,null,null);t.default=r.exports}}]); \ No newline at end of file diff --git a/assets/js/6.7b74ce4e.js b/assets/js/6.9cbfe13d.js similarity index 62% rename from assets/js/6.7b74ce4e.js rename to assets/js/6.9cbfe13d.js index ee7a08a6..c93c97c8 100644 --- a/assets/js/6.7b74ce4e.js +++ b/assets/js/6.9cbfe13d.js @@ -1 +1 @@ -(window.webpackJsonp=window.webpackJsonp||[]).push([[6],{339:function(t,e,n){},371:function(t,e,n){"use strict";n(339)},404:function(t,e,n){"use strict";n.r(e);var i={functional:!0,props:{type:{type:String,default:"tip"},text:String,vertical:{type:String,default:"top"}},render:function(t,e){var n=e.props,i=e.slots;return t("span",{class:["badge",n.type],style:{verticalAlign:n.vertical}},n.text||i().default)}},r=(n(371),n(26)),p=Object(r.a)(i,void 0,void 0,!1,null,"15b7b770",null);e.default=p.exports}}]); \ No newline at end of file +(window.webpackJsonp=window.webpackJsonp||[]).push([[6],{338:function(t,e,n){},370:function(t,e,n){"use strict";n(338)},404:function(t,e,n){"use strict";n.r(e);var i={functional:!0,props:{type:{type:String,default:"tip"},text:String,vertical:{type:String,default:"top"}},render:function(t,e){var n=e.props,i=e.slots;return t("span",{class:["badge",n.type],style:{verticalAlign:n.vertical}},n.text||i().default)}},r=(n(370),n(26)),p=Object(r.a)(i,void 0,void 0,!1,null,"15b7b770",null);e.default=p.exports}}]); \ No newline at end of file diff --git a/assets/js/7.31131f87.js b/assets/js/7.9ebea86b.js similarity index 67% rename from assets/js/7.31131f87.js rename to assets/js/7.9ebea86b.js index e8f102e5..aff6a79a 100644 --- a/assets/js/7.31131f87.js +++ b/assets/js/7.9ebea86b.js @@ -1 +1 @@ -(window.webpackJsonp=window.webpackJsonp||[]).push([[7],{338:function(t,e,a){},370:function(t,e,a){"use strict";a(338)},380:function(t,e,a){"use strict";a.r(e);var n={name:"CodeBlock",props:{title:{type:String,required:!0},active:{type:Boolean,default:!1}},mounted:function(){this.$parent&&this.$parent.loadTabs&&this.$parent.loadTabs()}},i=(a(370),a(26)),s=Object(i.a)(n,(function(){var t=this.$createElement;return(this._self._c||t)("div",{staticClass:"theme-code-block",class:{"theme-code-block__active":this.active}},[this._t("default")],2)}),[],!1,null,"759a7d02",null);e.default=s.exports}}]); \ No newline at end of file +(window.webpackJsonp=window.webpackJsonp||[]).push([[7],{339:function(t,e,a){},371:function(t,e,a){"use strict";a(339)},380:function(t,e,a){"use strict";a.r(e);var n={name:"CodeBlock",props:{title:{type:String,required:!0},active:{type:Boolean,default:!1}},mounted:function(){this.$parent&&this.$parent.loadTabs&&this.$parent.loadTabs()}},i=(a(371),a(26)),s=Object(i.a)(n,(function(){var t=this.$createElement;return(this._self._c||t)("div",{staticClass:"theme-code-block",class:{"theme-code-block__active":this.active}},[this._t("default")],2)}),[],!1,null,"759a7d02",null);e.default=s.exports}}]); \ No newline at end of file diff --git a/assets/js/9.1f4f5f7a.js b/assets/js/9.d491b2cb.js similarity index 99% rename from assets/js/9.1f4f5f7a.js rename to assets/js/9.d491b2cb.js index 3dd92e4e..e4ff091b 100644 --- a/assets/js/9.1f4f5f7a.js +++ b/assets/js/9.d491b2cb.js @@ -1 +1 @@ -(window.webpackJsonp=window.webpackJsonp||[]).push([[9],{335:function(t,e,a){},367:function(t,e,a){"use strict";a(335)},399:function(t,e,a){"use strict";a.r(e);a(367);var s=a(26),d=Object(s.a)({},(function(){var t=this,e=t.$createElement,a=t._self._c||e;return a("ContentSlotsDistributor",{attrs:{"slot-key":t.$parent.slotKey}},[a("h1",{attrs:{id:"page-frontmatter-title"}},[a("a",{staticClass:"header-anchor",attrs:{href:"#page-frontmatter-title"}},[t._v("#")]),t._v(" "+t._s(t.$page.frontmatter.title))]),t._v(" "),a("p",[t._v(t._s(t.$page.frontmatter.abstract))]),t._v(" "),a("MetadataTable"),t._v(" "),a("h2",{attrs:{id:"language"}},[a("a",{staticClass:"header-anchor",attrs:{href:"#language"}},[t._v("#")]),t._v(" Language")]),t._v(" "),a("Language"),t._v(" "),a("h2",{attrs:{id:"summary"}},[a("a",{staticClass:"header-anchor",attrs:{href:"#summary"}},[t._v("#")]),t._v(" Summary")]),t._v(" "),a("p",[t._v("The tabular diff format expresses the difference between two versions of a table."),a("br"),t._v("\nHere’s an example of a diff:")]),t._v(" "),a("div",{staticClass:"highlighter"},[a("table",[a("tr",[a("th",[t._v("@@")]),a("th",[t._v("bridge")]),a("th",[t._v("designer")]),a("th",[t._v("length")])]),t._v(" "),a("tr",[a("td"),a("td",[t._v("Brooklyn")]),a("td",[t._v("J. A. Roebling")]),a("td",[t._v("1595")])]),t._v(" "),a("tr",{staticClass:"add"},[a("td",[t._v("+++")]),a("td",[t._v("Manhattan")]),a("td",[t._v("G. Lindenthal")]),a("td",[t._v("1470")])]),t._v(" "),a("tr",{staticClass:"modify"},[a("td",{staticClass:"modify"},[t._v("->")]),a("td",[t._v("Williamsburg")]),a("td",{staticClass:"modify"},[t._v("D. Duck->L. L. Buck")]),a("td",[t._v("1600")])]),t._v(" "),a("tr",[a("td"),a("td",[t._v("Queensborough")]),a("td",[t._v("Palmer & Hornbostel")]),a("td",[t._v("1182")])]),t._v(" "),a("tr",[a("td",[t._v("...")]),a("td",[t._v("...")]),a("td",[t._v("...")]),a("td",[t._v("...")])]),t._v(" "),a("tr",[a("td"),a("td",[t._v("George Washington")]),a("td",[t._v("O. H. Ammann")]),a("td",[t._v("3500")])]),t._v(" "),a("tr",{staticClass:"remove"},[a("td",[t._v("---")]),a("td",[t._v("Spamspan")]),a("td",[t._v("S. Spamington")]),a("td",[t._v("10000 ")])])])]),t._v(" "),a("p",[t._v("As for text diffs, the format emphasizes significant changes. Also like text diffs, the format is unambiguous without color, but readily enhanced with it. Unlike text diffs, the format preserves the original tabular structure, allowing presentation with sensible visual alignment.")]),t._v(" "),a("p",[t._v("There is a reference implementation of a tool for generating and processing tabular diffs at "),a("a",{attrs:{href:"https://github.com/paulfitz/daff",target:"_blank",rel:"noopener noreferrer"}},[t._v("https://github.com/paulfitz/daff"),a("OutboundLink")],1),t._v(".")]),t._v(" "),a("h2",{attrs:{id:"scope"}},[a("a",{staticClass:"header-anchor",attrs:{href:"#scope"}},[t._v("#")]),t._v(" Scope")]),t._v(" "),a("p",[t._v("The tabular diff format can express the following kinds of changes in a table:")]),t._v(" "),a("ul",[a("li",[t._v("Inserted or deleted rows.")]),t._v(" "),a("li",[t._v("Inserted, deleted, or renamed columns.")]),t._v(" "),a("li",[t._v("Modified cell values.")])]),t._v(" "),a("p",[t._v("If the order of the rows or columns of the the table are meaningful, then the format can also express:")]),t._v(" "),a("ul",[a("li",[t._v("Reordered rows or columns.")])]),t._v(" "),a("p",[t._v("Changes in formatting and systematic transformation of data (such as capitalization) are not expressible.")]),t._v(" "),a("h2",{attrs:{id:"general-structure"}},[a("a",{staticClass:"header-anchor",attrs:{href:"#general-structure"}},[t._v("#")]),t._v(" General structure")]),t._v(" "),a("p",[t._v("Assume we have two tables, called "),a("code",[t._v("LOCAL")]),t._v(" and "),a("code",[t._v("REMOTE")]),t._v(". The diff summarizes the changes needed to modify "),a("code",[t._v("LOCAL")]),t._v(" to match "),a("code",[t._v("REMOTE")]),t._v(".")]),t._v(" "),a("p",[t._v("The diff contains rows and columns from the tables being compared. As in regular text diffs, there is flexibility in what data is given and what is omitted.")]),t._v(" "),a("ul",[a("li",[t._v("A column or row that is common to the tables being compared should appear at most once.")]),t._v(" "),a("li",[t._v("Any column or row containing a modified cell should be included in the diff, and the modified cell should be represented using the procedure in "),a("a",{attrs:{href:"#expressing-a-modified-cell-value"}},[t._v("Expressing a modified cell value")]),t._v(".")]),t._v(" "),a("li",[t._v("Columns or rows that are present in one table and not in the other should be included in the diff.")]),t._v(" "),a("li",[t._v("Columns or rows that are unchanged and unneeded for context may be omitted, at the diff creator’s discretion.")]),t._v(" "),a("li",[t._v("Omitted blocks of rows or columns should be marked with a row/column full of "),a("code",[t._v("...")]),t._v(" cells.")])]),t._v(" "),a("p",[t._v("In addition, the diff contains the following special rows and columns:")]),t._v(" "),a("ul",[a("li",[t._v("The "),a("em",[t._v("action")]),t._v(" column. This is always present, and is the first column in the diff if columns are ordered. If columns are "),a("em",[t._v("not")]),t._v(" ordered, it is the column named "),a("code",[t._v("__hilite_diff__")]),t._v(".")]),t._v(" "),a("li",[t._v("A "),a("em",[t._v("header")]),t._v(" row with column names. This row can be recognized since it will have the tag "),a("code",[t._v("@@")]),t._v(" in the action column.")]),t._v(" "),a("li",[t._v("A "),a("em",[t._v("schema")]),t._v(" row that is needed when the column structure differs between tables. This row can be recognized since it will have the tag "),a("code",[t._v("!")]),t._v(" in the action column.")])]),t._v(" "),a("p",[t._v("Here’s an example diff, where the tables being compared share the same three columns:")]),t._v(" "),a("div",{staticClass:"highlighter"},[a("table",[a("tr",{staticClass:"desc"},[a("td",{staticClass:"desc"}),a("td",[t._v("action column")]),a("td",{attrs:{colspan:"3"}},[t._v("data from compared tables")])]),t._v(" "),a("tr",[a("td",{staticClass:"desc"},[t._v("header row")]),a("th",[t._v("@@")]),a("th",[t._v("bridge")]),a("th",[t._v("designer")]),a("th",[t._v("length")])]),t._v(" "),a("tr",[a("td",{staticClass:"desc"}),a("td"),a("td",[t._v("Brooklyn")]),a("td",[t._v("J. A. Roebling")]),a("td",[t._v("1595")])]),t._v(" "),a("tr",{staticClass:"add"},[a("td",{staticClass:"desc"}),a("td",[t._v("+++")]),a("td",[t._v("Manhattan")]),a("td",[t._v("G. Lindenthal")]),a("td",[t._v("1470")])]),t._v(" "),a("tr",{staticClass:"modify"},[a("td",{staticClass:"desc"}),a("td",{staticClass:"modify"},[t._v("->")]),a("td",[t._v("Williamsburg")]),a("td",{staticClass:"modify"},[t._v("D. Duck->L. L. Buck")]),a("td",[t._v("1600")])]),t._v(" "),a("tr",[a("td",{staticClass:"desc"}),a("td"),a("td",[t._v("Queensborough")]),a("td",[t._v("Palmer & Hornbostel")]),a("td",[t._v("1182")])]),t._v(" "),a("tr",[a("td",{staticClass:"desc"},[t._v("omitted rows")]),a("td",[t._v("...")]),a("td",[t._v("...")]),a("td",[t._v("...")]),a("td",[t._v("...")])]),t._v(" "),a("tr",[a("td",{staticClass:"desc"}),a("td"),a("td",[t._v("George Washington")]),a("td",[t._v("O. H. Ammann")]),a("td",[t._v("3500")])]),t._v(" "),a("tr",{staticClass:"remove"},[a("td",{staticClass:"desc"}),a("td",[t._v("---")]),a("td",[t._v("Spamspan")]),a("td",[t._v("S. Spamington")]),a("td",[t._v("10000 ")])])])]),t._v(" "),a("p",[t._v("The colors do not make up part of this specification, they are just syntax highlighting. The meaning of the various tags will become clear in later sections, for now we are just concerned with the structure of the diff. Here’s an example where there is a difference in columns: "),a("code",[t._v("LOCAL")]),t._v(" has a "),a("em",[t._v("length")]),t._v(" column that is removed in "),a("code",[t._v("REMOTE")]),t._v(", "),a("code",[t._v("REMOTE")]),t._v(" has an "),a("em",[t._v("opened")]),t._v(" column that wasn’t present in "),a("code",[t._v("LOCAL")]),t._v(", and the "),a("em",[t._v("designer")]),t._v(" column in "),a("code",[t._v("LOCAL")]),t._v(" is renamed as "),a("em",[t._v("lead designer")]),t._v(" in "),a("code",[t._v("REMOTE")]),t._v(":")]),t._v(" "),a("div",{staticClass:"highlighter"},[a("table",[a("tr",{staticClass:"desc"},[a("td",{staticClass:"desc"}),a("td",[t._v("action column")]),a("td",{attrs:{colspan:"4"}},[t._v("data from compared tables")])]),t._v(" "),a("tr",{staticClass:"spec"},[a("td",{staticClass:"desc"},[t._v("schema row")]),a("td",[t._v("!")]),a("td"),a("td",{staticClass:"add"},[t._v("+++")]),a("td",[t._v("(designer)")]),a("td",{staticClass:"remove"},[t._v("---")])]),t._v(" "),a("tr",[a("td",{staticClass:"desc"},[t._v("header row")]),a("th",[t._v("@@")]),a("th",[t._v("bridge")]),a("th",{staticClass:"add"},[t._v("opened")]),a("th",[t._v("lead designer")]),a("th",{staticClass:"remove"},[t._v("length")])]),t._v(" "),a("tr",[a("td",{staticClass:"desc",attrs:{rowspan:"8"}}),a("td",[t._v("+")]),a("td",[t._v("Brooklyn")]),a("td",{staticClass:"add"},[t._v("1883")]),a("td",[t._v("J. A. Roebling")]),a("td",{staticClass:"remove"},[t._v("1595")])]),t._v(" "),a("tr",[a("td",[t._v("+")]),a("td",[t._v("Manhattan")]),a("td",{staticClass:"add"},[t._v("1909")]),a("td",[t._v("G. Lindenthal")]),a("td",{staticClass:"remove"},[t._v("1470")])]),t._v(" "),a("tr",[a("td",[t._v("+")]),a("td",[t._v("Williamsburg")]),a("td",{staticClass:"add"},[t._v("1903")]),a("td",[t._v("L. L. Buck")]),a("td",{staticClass:"remove"},[t._v("1600")])]),t._v(" "),a("tr",[a("td",[t._v("+")]),a("td",[t._v("Queensborough")]),a("td",{staticClass:"add"},[t._v("1909")]),a("td",[t._v("Palmer & Hornbostel")]),a("td",{staticClass:"remove"},[t._v("1182")])]),t._v(" "),a("tr",[a("td",[t._v("+")]),a("td",[t._v("Triborough")]),a("td",{staticClass:"add"},[t._v("1936")]),a("td",[t._v("O. H. Ammann")]),a("td",{staticClass:"remove"},[t._v("1380,383")])]),t._v(" "),a("tr",[a("td",[t._v("+")]),a("td",[t._v("Bronx Whitestone")]),a("td",{staticClass:"add"},[t._v("1939")]),a("td",[t._v("O. H. Ammann")]),a("td",{staticClass:"remove"},[t._v("2300")])]),t._v(" "),a("tr",[a("td",[t._v("+")]),a("td",[t._v("Throgs Neck")]),a("td",{staticClass:"add"},[t._v("1961")]),a("td",[t._v("O. H. Ammann")]),a("td",{staticClass:"remove"},[t._v("1800")])]),t._v(" "),a("tr",[a("td",[t._v("+")]),a("td",[t._v("George Washington")]),a("td",{staticClass:"add"},[t._v("1931")]),a("td",[t._v("O. H. Ammann")]),a("td",{staticClass:"remove"},[t._v("3500")])])])]),t._v(" "),a("p",[t._v("We see that a schema row is added above the header row to represent the changes in columns. With this general anatomy of a diff in mind, let’s look at the details of how to interpret it.")]),t._v(" "),a("div",{staticClass:"custom-block tip"},[a("p",{staticClass:"custom-block-title"},[t._v("TIP")]),t._v(" "),a("p",[t._v("If writing a rule to “sniff” a file to see if it is a tabular diff, the "),a("code",[t._v("@@")]),t._v(" tag is a handy tell-tale. But watch out for that schema row! Also, to allow for future evolution of this format, please try to be robust to a few extra rows or columns appearing before the "),a("code",[t._v("@@")]),t._v(".")])]),t._v(" "),a("h2",{attrs:{id:"expressing-inserted-and-deleted-columns"}},[a("a",{staticClass:"header-anchor",attrs:{href:"#expressing-inserted-and-deleted-columns"}},[t._v("#")]),t._v(" Expressing inserted and deleted columns")]),t._v(" "),a("p",[t._v("An inserted column is expressed simply by including that column in the diff, and placing "),a("code",[t._v("+++")]),t._v(" in the schema row above the corresponding column name in the header row. Similarly, a deleted column is expressed by including that column in the diff, and placing "),a("code",[t._v("---")]),t._v(" in the schema row above the corresponding column name. As a special case, a renamed column is represented by simply placing its old name in parentheses in the schema row.")]),t._v(" "),a("p",[t._v("In our earlier example, "),a("code",[t._v("LOCAL")]),t._v(" has the columns "),a("em",[t._v("bridge")]),t._v(", "),a("em",[t._v("designer")]),t._v(", and "),a("em",[t._v("length")]),t._v(", while "),a("code",[t._v("REMOTE")]),t._v(" has the columns "),a("em",[t._v("bridge")]),t._v(", "),a("em",[t._v("opened")]),t._v(", and "),a("em",[t._v("lead designer")]),t._v(" ("),a("em",[t._v("designer")]),t._v(" renamed). So "),a("em",[t._v("opened")]),t._v(" is inserted and "),a("em",[t._v("length")]),t._v(" is deleted:")]),t._v(" "),a("div",{staticClass:"highlighter"},[a("table",[a("tr",{staticClass:"desc"},[a("td",{staticClass:"desc"}),a("td",[t._v("action"),a("br"),t._v("column")]),a("td",{attrs:{colspan:"4"}},[t._v("data from compared tables")])]),t._v(" "),a("tr",{staticClass:"spec"},[a("td",{staticClass:"desc"},[t._v("schema row")]),a("td",[t._v("!")]),a("td"),a("td",{staticClass:"add"},[t._v("+++")]),a("td",[t._v("(designer)")]),a("td",{staticClass:"remove"},[t._v("---")])]),t._v(" "),a("tr",[a("td",{staticClass:"desc"},[t._v("header row")]),a("th",[t._v("@@")]),a("th",[t._v("bridge")]),a("th",{staticClass:"add"},[t._v("opened")]),a("th",[t._v("lead designer")]),a("th",{staticClass:"remove"},[t._v("length")])]),t._v(" "),a("tr",[a("td",{staticClass:"desc",attrs:{rowspan:"3"}}),a("td",[t._v("+")]),a("td",[t._v("Brooklyn")]),a("td",{staticClass:"add"},[t._v("1883")]),a("td",[t._v("J. A. Roebling")]),a("td",{staticClass:"remove"},[t._v("1595")])]),t._v(" "),a("tr",[a("td",[t._v("+")]),a("td",[t._v("Manhattan")]),a("td",{staticClass:"add"},[t._v("1909")]),a("td",[t._v("G. Lindenthal")]),a("td",{staticClass:"remove"},[t._v("1470")])]),t._v(" "),a("tr",[a("td",[t._v("+")]),a("td",[t._v("Williamsburg")]),a("td",{staticClass:"add"},[t._v("1903")]),a("td",[t._v("L. L. Buck")]),a("td",{staticClass:"remove"},[t._v("1600")])])])]),t._v(" "),a("p",[t._v("If we are dealing with a data store where columns are unordered, then likewise column order in the diff is irrelevant. Otherwise, the inserted and deleted rows should be placed in their appropriate order.")]),t._v(" "),a("p",[t._v("Any rows in the diff that are present only the "),a("code",[t._v("LOCAL")]),t._v(" table will leave inserted columns blank. Similarly, any rows in the diff that are present only in the "),a("code",[t._v("REMOTE")]),t._v(" table will leave deleted columns blank. Rows that are present in both tables will have values in all cells.")]),t._v(" "),a("h2",{attrs:{id:"expressing-inserted-and-deleted-rows"}},[a("a",{staticClass:"header-anchor",attrs:{href:"#expressing-inserted-and-deleted-rows"}},[t._v("#")]),t._v(" Expressing inserted and deleted rows")]),t._v(" "),a("p",[t._v("An inserted row is expressed simply by placing "),a("code",[t._v("+++")]),t._v(" in the action column, and placing cell values in the appropriate columns. If there are columns in the diff that are in "),a("code",[t._v("LOCAL")]),t._v(" but not in "),a("code",[t._v("REMOTE")]),t._v(", these are left blank. Likewise, a deleted row is expressed by placing "),a("code",[t._v("---")]),t._v(" in the action column, and its cell values in the appropriate columns. If there are columns in the diff that are in "),a("code",[t._v("REMOTE")]),t._v(" but not in "),a("code",[t._v("LOCAL")]),t._v(", these are left blank. For example, suppose in "),a("code",[t._v("REMOTE")]),t._v(" there is a row about a New Bridge that wasn’t in "),a("code",[t._v("LOCAL")]),t._v(", and a row about a bridge called Spamspan has been dropped. Here is what the inserted and deleted rows would look like, lined up with the header row for reference:")]),t._v(" "),a("div",{staticClass:"highlighter"},[a("table",[a("tr",{staticClass:"desc"},[a("td",{staticClass:"desc"}),a("td",[t._v("action"),a("br"),t._v("column")]),a("td",{attrs:{colspan:"4"}})]),t._v(" "),a("tr",{staticClass:"spec"},[a("td",{staticClass:"desc"},[t._v("schema row")]),a("td",[t._v("!")]),a("td"),a("td",{staticClass:"add"},[t._v("+++")]),a("td",[t._v("(designer)")]),a("td",{staticClass:"remove"},[t._v("---")])]),t._v(" "),a("tr",[a("td",{staticClass:"desc"},[t._v("header row")]),a("th",[t._v("@@")]),a("th",[t._v("bridge")]),a("th",{staticClass:"add"},[t._v("opened")]),a("th",[t._v("lead designer")]),a("th",{staticClass:"remove"},[t._v("length")])]),t._v(" "),a("tr",{staticClass:"blank"},[a("td",[t._v(" ")]),a("td"),a("td"),a("td"),a("td"),a("td")]),t._v(" "),a("tr",{staticClass:"add"},[a("td",{staticClass:"desc"},[t._v("inserted row")]),a("td",[t._v("+++")]),a("td",[t._v("New Bridge")]),a("td",{staticClass:"add"},[t._v("2050")]),a("td",[t._v("Chimp N Zee")]),a("td",{staticClass:"remove"},[t._v("   ")])]),t._v(" "),a("tr",{staticClass:"blank"},[a("td",[t._v(" ")]),a("td"),a("td"),a("td"),a("td"),a("td")]),t._v(" "),a("tr",{staticClass:"remove"},[a("td",{staticClass:"desc"},[t._v("deleted row")]),a("td",[t._v("---")]),a("td",[t._v("Spamspan")]),a("td"),a("td",[t._v("S. Spamington")]),a("td",{staticClass:"remove"},[t._v("10000")])])])]),t._v(" "),a("p",[t._v("If the diff is on a database table where rows have no ordering, then we can just stick these rows together and we have our diff:")]),t._v(" "),a("div",{staticClass:"highlighter"},[a("table",[a("tr",{staticClass:"desc"},[a("td",{staticClass:"desc"}),a("td",[t._v("action"),a("br"),t._v("column")]),a("td",{attrs:{colspan:"4"}})]),t._v(" "),a("tr",{staticClass:"spec"},[a("td",{staticClass:"desc"},[t._v("schema row")]),a("td",[t._v("!")]),a("td"),a("td",{staticClass:"add"},[t._v("+++")]),a("td",[t._v("(designer)")]),a("td",{staticClass:"remove"},[t._v("---")])]),t._v(" "),a("tr",[a("td",{staticClass:"desc"},[t._v("header row")]),a("th",[t._v("@@")]),a("th",[t._v("bridge")]),a("th",{staticClass:"add"},[t._v("opened")]),a("th",[t._v("lead designer")]),a("th",{staticClass:"remove"},[t._v("length")])]),t._v(" "),a("tr",{staticClass:"add"},[a("td",{staticClass:"desc"},[t._v("inserted row")]),a("td",[t._v("+++")]),a("td",[t._v("New Bridge")]),a("td",{staticClass:"add"},[t._v("2050")]),a("td",[t._v("Chimp N Zee")]),a("td",{staticClass:"remove"},[t._v("   ")])]),t._v(" "),a("tr",{staticClass:"remove"},[a("td",{staticClass:"desc"},[t._v("deleted row")]),a("td",[t._v("---")]),a("td",[t._v("Spamspan")]),a("td"),a("td",[t._v("S. Spamington")]),a("td",{staticClass:"remove"},[t._v("10000")])])])]),t._v(" "),a("p",[t._v("If the diff is on a spreadsheet table or CSV file, we’d generally want to respect row ordering. In this case, we can add context rows around insertions so we know where to put them. Less importantly, since they are going away anyway, we can do the same for deletions:")]),t._v(" "),a("div",{staticClass:"highlighter"},[a("table",[a("tr",{staticClass:"desc"},[a("td",{staticClass:"desc"}),a("td",[t._v("action"),a("br"),t._v("column")]),a("td",{attrs:{colspan:"4"}})]),t._v(" "),a("tr",{staticClass:"spec"},[a("td",{staticClass:"desc"},[t._v("schema row")]),a("td",[t._v("!")]),a("td"),a("td",{staticClass:"add"},[t._v("+++")]),a("td",[t._v("(designer)")]),a("td",{staticClass:"remove"},[t._v("---")])]),t._v(" "),a("tr",[a("td",{staticClass:"desc"},[t._v("header row")]),a("th",[t._v("@@")]),a("th",[t._v("bridge")]),a("th",{staticClass:"add"},[t._v("opened")]),a("th",[t._v("lead designer")]),a("th",{staticClass:"remove"},[t._v("length")])]),t._v(" "),a("tr",[a("td",{staticClass:"desc"},[t._v("omitted rows")]),a("td",[t._v("...")]),a("td",[t._v("...")]),a("td",[t._v("...")]),a("td",[t._v("...")]),a("td",[t._v("...")])]),t._v(" "),a("tr",[a("td",{staticClass:"desc"},[t._v("context row")]),a("td",[t._v("+")]),a("td",[t._v("Williamsburg")]),a("td",{staticClass:"add"},[t._v("1903")]),a("td",[t._v("L. L. Buck")]),a("td",{staticClass:"remove"},[t._v("1600")])]),t._v(" "),a("tr",{staticClass:"add"},[a("td",{staticClass:"desc"},[t._v("inserted row")]),a("td",[t._v("+++")]),a("td",[t._v("New Bridge")]),a("td",{staticClass:"add"},[t._v("2050")]),a("td",[t._v("Chimp N Zee")]),a("td",{staticClass:"remove"},[t._v("   ")])]),t._v(" "),a("tr",[a("td",{staticClass:"desc"},[t._v("context row")]),a("td",[t._v("+")]),a("td",[t._v("Queensborough")]),a("td",{staticClass:"add"},[t._v("1909")]),a("td",[t._v("Palmer & Hornbostel")]),a("td",{staticClass:"remove"},[t._v("1182")])]),t._v(" "),a("tr",[a("td",{staticClass:"desc"},[t._v("omitted rows")]),a("td",[t._v("...")]),a("td",[t._v("...")]),a("td",[t._v("...")]),a("td",[t._v("...")]),a("td",[t._v("...")])]),t._v(" "),a("tr",[a("td",{staticClass:"desc"},[t._v("context row")]),a("td",[t._v("+")]),a("td",[t._v("George Washington")]),a("td",{staticClass:"add"},[t._v("1931")]),a("td",[t._v("O. H. Ammann")]),a("td",{staticClass:"remove"},[t._v("3500")])]),t._v(" "),a("tr",{staticClass:"remove"},[a("td",{staticClass:"desc"},[t._v("deleted row")]),a("td",[t._v("---")]),a("td",[t._v("Spamspan")]),a("td"),a("td",[t._v("S. Spamington")]),a("td",{staticClass:"remove"},[t._v("10000")])])])]),t._v(" "),a("p",[t._v("The action column for a context row may contain a blank, or "),a("code",[t._v(":")]),t._v(", or "),a("code",[t._v("+")]),t._v(". The "),a("code",[t._v(":")]),t._v(" tag signifies the context row was moved (and its location is now as in the "),a("code",[t._v("REMOTE")]),t._v(" table). The "),a("code",[t._v("+")]),t._v(" signifies that there are cells added on that row.")]),t._v(" "),a("h2",{attrs:{id:"expressing-a-modified-cell-value"}},[a("a",{staticClass:"header-anchor",attrs:{href:"#expressing-a-modified-cell-value"}},[t._v("#")]),t._v(" Expressing a modified cell value")]),t._v(" "),a("p",[t._v("If a row contains a cell whose value is different in the compared tables, then that row should be shown in the diff, with a tag in the action column that ends in "),a("code",[t._v("->")]),t._v(". Then, the modified cell will be represented by converting the "),a("code",[t._v("LOCAL")]),t._v(" and "),a("code",[t._v("REMOTE")]),t._v(" values to text (we have yet to say how) and using the action tag as a separator. So for example here we change the last cell in a row from “Green” to “Blue”:")]),t._v(" "),a("div",{staticClass:"highlighter"},[a("table",[a("tr",[a("td",{staticClass:"modify"},[t._v("->")]),a("td",[t._v("Gnome")]),a("td",[t._v("Home and Garden")]),a("td",{staticClass:"modify"},[t._v("Green->Blue")])])])]),t._v(" "),a("p",[t._v("The tag must be preceded with as many extra "),a("code",[t._v("-")]),t._v(" characters as are needed to avoid collision with any character sequence on that row. So here is another row with exactly one cell changed:")]),t._v(" "),a("div",{staticClass:"highlighter"},[a("table",[a("tr",[a("td",{staticClass:"modify"},[t._v("--\x3e")]),a("td",[t._v("Console")]),a("td",[t._v("Toddlers -> Teenagers")]),a("td",{staticClass:"modify"},[t._v("White--\x3ePale")])])])]),t._v(" "),a("p",[t._v("When encoding a cell change as a string, we lose information about the type of the cell value. One distinction that may be important to retain is the difference between a NULL or empty cell, and the empty string. The tabular diff uses the following encoding:")]),t._v(" "),a("ul",[a("li",[t._v("A NULL value, if available, represents itself.")]),t._v(" "),a("li",[t._v("The encoded string "),a("code",[t._v("NULL")]),t._v(" represents a NULL value.")]),t._v(" "),a("li",[t._v("The encoded string "),a("code",[t._v("_NULL")]),t._v(" represents the string “NULL”.")]),t._v(" "),a("li",[t._v("The encoded string "),a("code",[t._v("__NULL")]),t._v(" represents the string “_NULL”.")]),t._v(" "),a("li",[t._v("…")])]),t._v(" "),a("p",[t._v("The goal is that the diff can be safely converted to and from CSV by existing tools without changing its meaning. To that end:")]),t._v(" "),a("ul",[a("li",[t._v("For matching (e.g. on context lines) blank cells in the diff (either the NULL value or an empty string) should be treated as ambiguous, and match "),a("em",[t._v("either")]),t._v(" of NULL or an empty string if an exact match is not available.")]),t._v(" "),a("li",[t._v("When using a diff as a patch, and inserting new cells, a blank cell in the diff (either the NULL value or an empty string) should be treated as ambiguous, and the “right” thing done given the column type. If either value could be inserted, then the blank string should be inserted (since the encoded string "),a("code",[t._v("NULL")]),t._v(" is available to specifically identify the NULL value).")])]),t._v(" "),a("p",[t._v("Note that if the diff is being expressed in a table that allows nested structure (e.g. a JSON representation), a list representation for modified cells might be used to avoid this issue. There is no specification for that at this time.")]),t._v(" "),a("h2",{attrs:{id:"expressing-a-moved-row"}},[a("a",{staticClass:"header-anchor",attrs:{href:"#expressing-a-moved-row"}},[t._v("#")]),t._v(" Expressing a moved row")]),t._v(" "),a("div",{staticClass:"custom-block tip"},[a("p",{staticClass:"custom-block-title"},[t._v("TIP")]),t._v(" "),a("p",[t._v("This can be ignored for tables for which row order is meaningless, e.g. in typical relational databases.")])]),t._v(" "),a("p",[t._v("A row that have been moved in a table for which row order is meaningful is marked with a "),a("code",[t._v(":")]),t._v(" tag in the action column and placed in the order it appears in the "),a("code",[t._v("REMOTE")]),t._v(" table.")]),t._v(" "),a("p",[t._v("To avoid burdening human readers with too much arcana, tags are "),a("em",[t._v("not")]),t._v(" combined when multiple kinds of actions apply to a row or column. So for example, a context row that was moved and had a cell added will "),a("em",[t._v("not")]),t._v(" be tagged as "),a("code",[t._v(":+")]),t._v(" or "),a("code",[t._v("+:")]),t._v(" or such-like, but rather by "),a("code",[t._v(":")]),t._v(". Cell addition can be determined from the schema row. These weak tags are included as aids for highlighting to express the most significant thing to know about a row.")]),t._v(" "),a("h2",{attrs:{id:"expressing-a-moved-column"}},[a("a",{staticClass:"header-anchor",attrs:{href:"#expressing-a-moved-column"}},[t._v("#")]),t._v(" Expressing a moved column")]),t._v(" "),a("div",{staticClass:"custom-block tip"},[a("p",{staticClass:"custom-block-title"},[t._v("TIP")]),t._v(" "),a("p",[t._v("This can be ignored for tables for which column order is meaningless.")])]),t._v(" "),a("p",[t._v("A column that have been moved in a table for which column order is meaningful is marked with a "),a("code",[t._v(":")]),t._v(" tag in the schema row and placed in the order it appears in the "),a("code",[t._v("REMOTE")]),t._v(" table.")]),t._v(" "),a("p",[t._v("If a diff that contains a "),a("code",[t._v(":")]),t._v(" tag is used to patch a table for which column order is not meaningful, that tag should simply be ignored.")]),t._v(" "),a("h2",{attrs:{id:"reference-action-column-tags"}},[a("a",{staticClass:"header-anchor",attrs:{href:"#reference-action-column-tags"}},[t._v("#")]),t._v(" Reference: action column tags")]),t._v(" "),a("table",[a("thead",[a("tr",[a("th",[t._v("Symbol")]),t._v(" "),a("th",[t._v("Meaning")])])]),t._v(" "),a("tbody",[a("tr",[a("td",[a("code",[t._v("@@")])]),t._v(" "),a("td",[t._v("The header row, giving column names.")])]),t._v(" "),a("tr",[a("td",[a("code",[t._v("!")])]),t._v(" "),a("td",[t._v("The schema row, given column differences.")])]),t._v(" "),a("tr",[a("td",[a("code",[t._v("+++")])]),t._v(" "),a("td",[t._v("An inserted row (present in "),a("code",[t._v("REMOTE")]),t._v(", not present in "),a("code",[t._v("LOCAL")]),t._v(").")])]),t._v(" "),a("tr",[a("td",[a("code",[t._v("---")])]),t._v(" "),a("td",[t._v("A deleted row (present in "),a("code",[t._v("LOCAL")]),t._v(", not present in "),a("code",[t._v("REMOTE")]),t._v(").")])]),t._v(" "),a("tr",[a("td",[a("code",[t._v("->")])]),t._v(" "),a("td",[t._v("A row with at least one cell modified cell. "),a("code",[t._v("--\x3e")]),t._v(", "),a("code",[t._v("---\x3e")]),t._v(", "),a("code",[t._v("----\x3e")]),t._v(" etc. have the same meaning.")])]),t._v(" "),a("tr",[a("td",[t._v("Blank")]),t._v(" "),a("td",[t._v("A blank string or NULL marks a row common to "),a("code",[t._v("LOCAL")]),t._v(" and "),a("code",[t._v("REMOTE")]),t._v(", given for context.")])]),t._v(" "),a("tr",[a("td",[a("code",[t._v("...")])]),t._v(" "),a("td",[t._v("Declares that rows common to "),a("code",[t._v("LOCAL")]),t._v(" and "),a("code",[t._v("REMOTE")]),t._v(" are being skipped.")])]),t._v(" "),a("tr",[a("td",[a("code",[t._v("+")])]),t._v(" "),a("td",[t._v("A row with at least one added cell.")])]),t._v(" "),a("tr",[a("td",[a("code",[t._v(":")])]),t._v(" "),a("td",[t._v("A reordered row.")])])])]),t._v(" "),a("h2",{attrs:{id:"reference-schema-row-tags"}},[a("a",{staticClass:"header-anchor",attrs:{href:"#reference-schema-row-tags"}},[t._v("#")]),t._v(" Reference: schema row tags")]),t._v(" "),a("table",[a("thead",[a("tr",[a("th",[t._v("Symbol")]),t._v(" "),a("th",[t._v("Meaning")])])]),t._v(" "),a("tbody",[a("tr",[a("td",[a("code",[t._v("+++")])]),t._v(" "),a("td",[t._v("An inserted column (present in "),a("code",[t._v("REMOTE")]),t._v(", not present in "),a("code",[t._v("LOCAL")]),t._v(").")])]),t._v(" "),a("tr",[a("td",[a("code",[t._v("---")])]),t._v(" "),a("td",[t._v("A deleted column (present in "),a("code",[t._v("LOCAL")]),t._v(", not present in "),a("code",[t._v("REMOTE")]),t._v(").")])]),t._v(" "),a("tr",[a("td",[a("code",[t._v("()")])]),t._v(" "),a("td",[t._v("A renamed column (the name in "),a("code",[t._v("LOCAL")]),t._v(" is given in parenthesis, and the name in "),a("code",[t._v("REMOTE")]),t._v(" will be in the header row).")])]),t._v(" "),a("tr",[a("td",[t._v("Blank")]),t._v(" "),a("td",[t._v("A blank string or NULL marks a column common to "),a("code",[t._v("LOCAL")]),t._v(" and "),a("code",[t._v("REMOTE")]),t._v(", given for context.")])]),t._v(" "),a("tr",[a("td",[a("code",[t._v("...")])]),t._v(" "),a("td",[t._v("Declares that columns common to "),a("code",[t._v("LOCAL")]),t._v(" and "),a("code",[t._v("REMOTE")]),t._v(" are being skipped.")])]),t._v(" "),a("tr",[a("td",[a("code",[t._v(":")])]),t._v(" "),a("td",[t._v("A reordered column.")])])])])],1)}),[],!1,null,null,null);e.default=d.exports}}]); \ No newline at end of file +(window.webpackJsonp=window.webpackJsonp||[]).push([[9],{335:function(t,e,a){},367:function(t,e,a){"use strict";a(335)},400:function(t,e,a){"use strict";a.r(e);a(367);var s=a(26),d=Object(s.a)({},(function(){var t=this,e=t.$createElement,a=t._self._c||e;return a("ContentSlotsDistributor",{attrs:{"slot-key":t.$parent.slotKey}},[a("h1",{attrs:{id:"page-frontmatter-title"}},[a("a",{staticClass:"header-anchor",attrs:{href:"#page-frontmatter-title"}},[t._v("#")]),t._v(" "+t._s(t.$page.frontmatter.title))]),t._v(" "),a("p",[t._v(t._s(t.$page.frontmatter.abstract))]),t._v(" "),a("MetadataTable"),t._v(" "),a("h2",{attrs:{id:"language"}},[a("a",{staticClass:"header-anchor",attrs:{href:"#language"}},[t._v("#")]),t._v(" Language")]),t._v(" "),a("Language"),t._v(" "),a("h2",{attrs:{id:"summary"}},[a("a",{staticClass:"header-anchor",attrs:{href:"#summary"}},[t._v("#")]),t._v(" Summary")]),t._v(" "),a("p",[t._v("The tabular diff format expresses the difference between two versions of a table."),a("br"),t._v("\nHere’s an example of a diff:")]),t._v(" "),a("div",{staticClass:"highlighter"},[a("table",[a("tr",[a("th",[t._v("@@")]),a("th",[t._v("bridge")]),a("th",[t._v("designer")]),a("th",[t._v("length")])]),t._v(" "),a("tr",[a("td"),a("td",[t._v("Brooklyn")]),a("td",[t._v("J. A. Roebling")]),a("td",[t._v("1595")])]),t._v(" "),a("tr",{staticClass:"add"},[a("td",[t._v("+++")]),a("td",[t._v("Manhattan")]),a("td",[t._v("G. Lindenthal")]),a("td",[t._v("1470")])]),t._v(" "),a("tr",{staticClass:"modify"},[a("td",{staticClass:"modify"},[t._v("->")]),a("td",[t._v("Williamsburg")]),a("td",{staticClass:"modify"},[t._v("D. Duck->L. L. Buck")]),a("td",[t._v("1600")])]),t._v(" "),a("tr",[a("td"),a("td",[t._v("Queensborough")]),a("td",[t._v("Palmer & Hornbostel")]),a("td",[t._v("1182")])]),t._v(" "),a("tr",[a("td",[t._v("...")]),a("td",[t._v("...")]),a("td",[t._v("...")]),a("td",[t._v("...")])]),t._v(" "),a("tr",[a("td"),a("td",[t._v("George Washington")]),a("td",[t._v("O. H. Ammann")]),a("td",[t._v("3500")])]),t._v(" "),a("tr",{staticClass:"remove"},[a("td",[t._v("---")]),a("td",[t._v("Spamspan")]),a("td",[t._v("S. Spamington")]),a("td",[t._v("10000 ")])])])]),t._v(" "),a("p",[t._v("As for text diffs, the format emphasizes significant changes. Also like text diffs, the format is unambiguous without color, but readily enhanced with it. Unlike text diffs, the format preserves the original tabular structure, allowing presentation with sensible visual alignment.")]),t._v(" "),a("p",[t._v("There is a reference implementation of a tool for generating and processing tabular diffs at "),a("a",{attrs:{href:"https://github.com/paulfitz/daff",target:"_blank",rel:"noopener noreferrer"}},[t._v("https://github.com/paulfitz/daff"),a("OutboundLink")],1),t._v(".")]),t._v(" "),a("h2",{attrs:{id:"scope"}},[a("a",{staticClass:"header-anchor",attrs:{href:"#scope"}},[t._v("#")]),t._v(" Scope")]),t._v(" "),a("p",[t._v("The tabular diff format can express the following kinds of changes in a table:")]),t._v(" "),a("ul",[a("li",[t._v("Inserted or deleted rows.")]),t._v(" "),a("li",[t._v("Inserted, deleted, or renamed columns.")]),t._v(" "),a("li",[t._v("Modified cell values.")])]),t._v(" "),a("p",[t._v("If the order of the rows or columns of the the table are meaningful, then the format can also express:")]),t._v(" "),a("ul",[a("li",[t._v("Reordered rows or columns.")])]),t._v(" "),a("p",[t._v("Changes in formatting and systematic transformation of data (such as capitalization) are not expressible.")]),t._v(" "),a("h2",{attrs:{id:"general-structure"}},[a("a",{staticClass:"header-anchor",attrs:{href:"#general-structure"}},[t._v("#")]),t._v(" General structure")]),t._v(" "),a("p",[t._v("Assume we have two tables, called "),a("code",[t._v("LOCAL")]),t._v(" and "),a("code",[t._v("REMOTE")]),t._v(". The diff summarizes the changes needed to modify "),a("code",[t._v("LOCAL")]),t._v(" to match "),a("code",[t._v("REMOTE")]),t._v(".")]),t._v(" "),a("p",[t._v("The diff contains rows and columns from the tables being compared. As in regular text diffs, there is flexibility in what data is given and what is omitted.")]),t._v(" "),a("ul",[a("li",[t._v("A column or row that is common to the tables being compared should appear at most once.")]),t._v(" "),a("li",[t._v("Any column or row containing a modified cell should be included in the diff, and the modified cell should be represented using the procedure in "),a("a",{attrs:{href:"#expressing-a-modified-cell-value"}},[t._v("Expressing a modified cell value")]),t._v(".")]),t._v(" "),a("li",[t._v("Columns or rows that are present in one table and not in the other should be included in the diff.")]),t._v(" "),a("li",[t._v("Columns or rows that are unchanged and unneeded for context may be omitted, at the diff creator’s discretion.")]),t._v(" "),a("li",[t._v("Omitted blocks of rows or columns should be marked with a row/column full of "),a("code",[t._v("...")]),t._v(" cells.")])]),t._v(" "),a("p",[t._v("In addition, the diff contains the following special rows and columns:")]),t._v(" "),a("ul",[a("li",[t._v("The "),a("em",[t._v("action")]),t._v(" column. This is always present, and is the first column in the diff if columns are ordered. If columns are "),a("em",[t._v("not")]),t._v(" ordered, it is the column named "),a("code",[t._v("__hilite_diff__")]),t._v(".")]),t._v(" "),a("li",[t._v("A "),a("em",[t._v("header")]),t._v(" row with column names. This row can be recognized since it will have the tag "),a("code",[t._v("@@")]),t._v(" in the action column.")]),t._v(" "),a("li",[t._v("A "),a("em",[t._v("schema")]),t._v(" row that is needed when the column structure differs between tables. This row can be recognized since it will have the tag "),a("code",[t._v("!")]),t._v(" in the action column.")])]),t._v(" "),a("p",[t._v("Here’s an example diff, where the tables being compared share the same three columns:")]),t._v(" "),a("div",{staticClass:"highlighter"},[a("table",[a("tr",{staticClass:"desc"},[a("td",{staticClass:"desc"}),a("td",[t._v("action column")]),a("td",{attrs:{colspan:"3"}},[t._v("data from compared tables")])]),t._v(" "),a("tr",[a("td",{staticClass:"desc"},[t._v("header row")]),a("th",[t._v("@@")]),a("th",[t._v("bridge")]),a("th",[t._v("designer")]),a("th",[t._v("length")])]),t._v(" "),a("tr",[a("td",{staticClass:"desc"}),a("td"),a("td",[t._v("Brooklyn")]),a("td",[t._v("J. A. Roebling")]),a("td",[t._v("1595")])]),t._v(" "),a("tr",{staticClass:"add"},[a("td",{staticClass:"desc"}),a("td",[t._v("+++")]),a("td",[t._v("Manhattan")]),a("td",[t._v("G. Lindenthal")]),a("td",[t._v("1470")])]),t._v(" "),a("tr",{staticClass:"modify"},[a("td",{staticClass:"desc"}),a("td",{staticClass:"modify"},[t._v("->")]),a("td",[t._v("Williamsburg")]),a("td",{staticClass:"modify"},[t._v("D. Duck->L. L. Buck")]),a("td",[t._v("1600")])]),t._v(" "),a("tr",[a("td",{staticClass:"desc"}),a("td"),a("td",[t._v("Queensborough")]),a("td",[t._v("Palmer & Hornbostel")]),a("td",[t._v("1182")])]),t._v(" "),a("tr",[a("td",{staticClass:"desc"},[t._v("omitted rows")]),a("td",[t._v("...")]),a("td",[t._v("...")]),a("td",[t._v("...")]),a("td",[t._v("...")])]),t._v(" "),a("tr",[a("td",{staticClass:"desc"}),a("td"),a("td",[t._v("George Washington")]),a("td",[t._v("O. H. Ammann")]),a("td",[t._v("3500")])]),t._v(" "),a("tr",{staticClass:"remove"},[a("td",{staticClass:"desc"}),a("td",[t._v("---")]),a("td",[t._v("Spamspan")]),a("td",[t._v("S. Spamington")]),a("td",[t._v("10000 ")])])])]),t._v(" "),a("p",[t._v("The colors do not make up part of this specification, they are just syntax highlighting. The meaning of the various tags will become clear in later sections, for now we are just concerned with the structure of the diff. Here’s an example where there is a difference in columns: "),a("code",[t._v("LOCAL")]),t._v(" has a "),a("em",[t._v("length")]),t._v(" column that is removed in "),a("code",[t._v("REMOTE")]),t._v(", "),a("code",[t._v("REMOTE")]),t._v(" has an "),a("em",[t._v("opened")]),t._v(" column that wasn’t present in "),a("code",[t._v("LOCAL")]),t._v(", and the "),a("em",[t._v("designer")]),t._v(" column in "),a("code",[t._v("LOCAL")]),t._v(" is renamed as "),a("em",[t._v("lead designer")]),t._v(" in "),a("code",[t._v("REMOTE")]),t._v(":")]),t._v(" "),a("div",{staticClass:"highlighter"},[a("table",[a("tr",{staticClass:"desc"},[a("td",{staticClass:"desc"}),a("td",[t._v("action column")]),a("td",{attrs:{colspan:"4"}},[t._v("data from compared tables")])]),t._v(" "),a("tr",{staticClass:"spec"},[a("td",{staticClass:"desc"},[t._v("schema row")]),a("td",[t._v("!")]),a("td"),a("td",{staticClass:"add"},[t._v("+++")]),a("td",[t._v("(designer)")]),a("td",{staticClass:"remove"},[t._v("---")])]),t._v(" "),a("tr",[a("td",{staticClass:"desc"},[t._v("header row")]),a("th",[t._v("@@")]),a("th",[t._v("bridge")]),a("th",{staticClass:"add"},[t._v("opened")]),a("th",[t._v("lead designer")]),a("th",{staticClass:"remove"},[t._v("length")])]),t._v(" "),a("tr",[a("td",{staticClass:"desc",attrs:{rowspan:"8"}}),a("td",[t._v("+")]),a("td",[t._v("Brooklyn")]),a("td",{staticClass:"add"},[t._v("1883")]),a("td",[t._v("J. A. Roebling")]),a("td",{staticClass:"remove"},[t._v("1595")])]),t._v(" "),a("tr",[a("td",[t._v("+")]),a("td",[t._v("Manhattan")]),a("td",{staticClass:"add"},[t._v("1909")]),a("td",[t._v("G. Lindenthal")]),a("td",{staticClass:"remove"},[t._v("1470")])]),t._v(" "),a("tr",[a("td",[t._v("+")]),a("td",[t._v("Williamsburg")]),a("td",{staticClass:"add"},[t._v("1903")]),a("td",[t._v("L. L. Buck")]),a("td",{staticClass:"remove"},[t._v("1600")])]),t._v(" "),a("tr",[a("td",[t._v("+")]),a("td",[t._v("Queensborough")]),a("td",{staticClass:"add"},[t._v("1909")]),a("td",[t._v("Palmer & Hornbostel")]),a("td",{staticClass:"remove"},[t._v("1182")])]),t._v(" "),a("tr",[a("td",[t._v("+")]),a("td",[t._v("Triborough")]),a("td",{staticClass:"add"},[t._v("1936")]),a("td",[t._v("O. H. Ammann")]),a("td",{staticClass:"remove"},[t._v("1380,383")])]),t._v(" "),a("tr",[a("td",[t._v("+")]),a("td",[t._v("Bronx Whitestone")]),a("td",{staticClass:"add"},[t._v("1939")]),a("td",[t._v("O. H. Ammann")]),a("td",{staticClass:"remove"},[t._v("2300")])]),t._v(" "),a("tr",[a("td",[t._v("+")]),a("td",[t._v("Throgs Neck")]),a("td",{staticClass:"add"},[t._v("1961")]),a("td",[t._v("O. H. Ammann")]),a("td",{staticClass:"remove"},[t._v("1800")])]),t._v(" "),a("tr",[a("td",[t._v("+")]),a("td",[t._v("George Washington")]),a("td",{staticClass:"add"},[t._v("1931")]),a("td",[t._v("O. H. Ammann")]),a("td",{staticClass:"remove"},[t._v("3500")])])])]),t._v(" "),a("p",[t._v("We see that a schema row is added above the header row to represent the changes in columns. With this general anatomy of a diff in mind, let’s look at the details of how to interpret it.")]),t._v(" "),a("div",{staticClass:"custom-block tip"},[a("p",{staticClass:"custom-block-title"},[t._v("TIP")]),t._v(" "),a("p",[t._v("If writing a rule to “sniff” a file to see if it is a tabular diff, the "),a("code",[t._v("@@")]),t._v(" tag is a handy tell-tale. But watch out for that schema row! Also, to allow for future evolution of this format, please try to be robust to a few extra rows or columns appearing before the "),a("code",[t._v("@@")]),t._v(".")])]),t._v(" "),a("h2",{attrs:{id:"expressing-inserted-and-deleted-columns"}},[a("a",{staticClass:"header-anchor",attrs:{href:"#expressing-inserted-and-deleted-columns"}},[t._v("#")]),t._v(" Expressing inserted and deleted columns")]),t._v(" "),a("p",[t._v("An inserted column is expressed simply by including that column in the diff, and placing "),a("code",[t._v("+++")]),t._v(" in the schema row above the corresponding column name in the header row. Similarly, a deleted column is expressed by including that column in the diff, and placing "),a("code",[t._v("---")]),t._v(" in the schema row above the corresponding column name. As a special case, a renamed column is represented by simply placing its old name in parentheses in the schema row.")]),t._v(" "),a("p",[t._v("In our earlier example, "),a("code",[t._v("LOCAL")]),t._v(" has the columns "),a("em",[t._v("bridge")]),t._v(", "),a("em",[t._v("designer")]),t._v(", and "),a("em",[t._v("length")]),t._v(", while "),a("code",[t._v("REMOTE")]),t._v(" has the columns "),a("em",[t._v("bridge")]),t._v(", "),a("em",[t._v("opened")]),t._v(", and "),a("em",[t._v("lead designer")]),t._v(" ("),a("em",[t._v("designer")]),t._v(" renamed). So "),a("em",[t._v("opened")]),t._v(" is inserted and "),a("em",[t._v("length")]),t._v(" is deleted:")]),t._v(" "),a("div",{staticClass:"highlighter"},[a("table",[a("tr",{staticClass:"desc"},[a("td",{staticClass:"desc"}),a("td",[t._v("action"),a("br"),t._v("column")]),a("td",{attrs:{colspan:"4"}},[t._v("data from compared tables")])]),t._v(" "),a("tr",{staticClass:"spec"},[a("td",{staticClass:"desc"},[t._v("schema row")]),a("td",[t._v("!")]),a("td"),a("td",{staticClass:"add"},[t._v("+++")]),a("td",[t._v("(designer)")]),a("td",{staticClass:"remove"},[t._v("---")])]),t._v(" "),a("tr",[a("td",{staticClass:"desc"},[t._v("header row")]),a("th",[t._v("@@")]),a("th",[t._v("bridge")]),a("th",{staticClass:"add"},[t._v("opened")]),a("th",[t._v("lead designer")]),a("th",{staticClass:"remove"},[t._v("length")])]),t._v(" "),a("tr",[a("td",{staticClass:"desc",attrs:{rowspan:"3"}}),a("td",[t._v("+")]),a("td",[t._v("Brooklyn")]),a("td",{staticClass:"add"},[t._v("1883")]),a("td",[t._v("J. A. Roebling")]),a("td",{staticClass:"remove"},[t._v("1595")])]),t._v(" "),a("tr",[a("td",[t._v("+")]),a("td",[t._v("Manhattan")]),a("td",{staticClass:"add"},[t._v("1909")]),a("td",[t._v("G. Lindenthal")]),a("td",{staticClass:"remove"},[t._v("1470")])]),t._v(" "),a("tr",[a("td",[t._v("+")]),a("td",[t._v("Williamsburg")]),a("td",{staticClass:"add"},[t._v("1903")]),a("td",[t._v("L. L. Buck")]),a("td",{staticClass:"remove"},[t._v("1600")])])])]),t._v(" "),a("p",[t._v("If we are dealing with a data store where columns are unordered, then likewise column order in the diff is irrelevant. Otherwise, the inserted and deleted rows should be placed in their appropriate order.")]),t._v(" "),a("p",[t._v("Any rows in the diff that are present only the "),a("code",[t._v("LOCAL")]),t._v(" table will leave inserted columns blank. Similarly, any rows in the diff that are present only in the "),a("code",[t._v("REMOTE")]),t._v(" table will leave deleted columns blank. Rows that are present in both tables will have values in all cells.")]),t._v(" "),a("h2",{attrs:{id:"expressing-inserted-and-deleted-rows"}},[a("a",{staticClass:"header-anchor",attrs:{href:"#expressing-inserted-and-deleted-rows"}},[t._v("#")]),t._v(" Expressing inserted and deleted rows")]),t._v(" "),a("p",[t._v("An inserted row is expressed simply by placing "),a("code",[t._v("+++")]),t._v(" in the action column, and placing cell values in the appropriate columns. If there are columns in the diff that are in "),a("code",[t._v("LOCAL")]),t._v(" but not in "),a("code",[t._v("REMOTE")]),t._v(", these are left blank. Likewise, a deleted row is expressed by placing "),a("code",[t._v("---")]),t._v(" in the action column, and its cell values in the appropriate columns. If there are columns in the diff that are in "),a("code",[t._v("REMOTE")]),t._v(" but not in "),a("code",[t._v("LOCAL")]),t._v(", these are left blank. For example, suppose in "),a("code",[t._v("REMOTE")]),t._v(" there is a row about a New Bridge that wasn’t in "),a("code",[t._v("LOCAL")]),t._v(", and a row about a bridge called Spamspan has been dropped. Here is what the inserted and deleted rows would look like, lined up with the header row for reference:")]),t._v(" "),a("div",{staticClass:"highlighter"},[a("table",[a("tr",{staticClass:"desc"},[a("td",{staticClass:"desc"}),a("td",[t._v("action"),a("br"),t._v("column")]),a("td",{attrs:{colspan:"4"}})]),t._v(" "),a("tr",{staticClass:"spec"},[a("td",{staticClass:"desc"},[t._v("schema row")]),a("td",[t._v("!")]),a("td"),a("td",{staticClass:"add"},[t._v("+++")]),a("td",[t._v("(designer)")]),a("td",{staticClass:"remove"},[t._v("---")])]),t._v(" "),a("tr",[a("td",{staticClass:"desc"},[t._v("header row")]),a("th",[t._v("@@")]),a("th",[t._v("bridge")]),a("th",{staticClass:"add"},[t._v("opened")]),a("th",[t._v("lead designer")]),a("th",{staticClass:"remove"},[t._v("length")])]),t._v(" "),a("tr",{staticClass:"blank"},[a("td",[t._v(" ")]),a("td"),a("td"),a("td"),a("td"),a("td")]),t._v(" "),a("tr",{staticClass:"add"},[a("td",{staticClass:"desc"},[t._v("inserted row")]),a("td",[t._v("+++")]),a("td",[t._v("New Bridge")]),a("td",{staticClass:"add"},[t._v("2050")]),a("td",[t._v("Chimp N Zee")]),a("td",{staticClass:"remove"},[t._v("   ")])]),t._v(" "),a("tr",{staticClass:"blank"},[a("td",[t._v(" ")]),a("td"),a("td"),a("td"),a("td"),a("td")]),t._v(" "),a("tr",{staticClass:"remove"},[a("td",{staticClass:"desc"},[t._v("deleted row")]),a("td",[t._v("---")]),a("td",[t._v("Spamspan")]),a("td"),a("td",[t._v("S. Spamington")]),a("td",{staticClass:"remove"},[t._v("10000")])])])]),t._v(" "),a("p",[t._v("If the diff is on a database table where rows have no ordering, then we can just stick these rows together and we have our diff:")]),t._v(" "),a("div",{staticClass:"highlighter"},[a("table",[a("tr",{staticClass:"desc"},[a("td",{staticClass:"desc"}),a("td",[t._v("action"),a("br"),t._v("column")]),a("td",{attrs:{colspan:"4"}})]),t._v(" "),a("tr",{staticClass:"spec"},[a("td",{staticClass:"desc"},[t._v("schema row")]),a("td",[t._v("!")]),a("td"),a("td",{staticClass:"add"},[t._v("+++")]),a("td",[t._v("(designer)")]),a("td",{staticClass:"remove"},[t._v("---")])]),t._v(" "),a("tr",[a("td",{staticClass:"desc"},[t._v("header row")]),a("th",[t._v("@@")]),a("th",[t._v("bridge")]),a("th",{staticClass:"add"},[t._v("opened")]),a("th",[t._v("lead designer")]),a("th",{staticClass:"remove"},[t._v("length")])]),t._v(" "),a("tr",{staticClass:"add"},[a("td",{staticClass:"desc"},[t._v("inserted row")]),a("td",[t._v("+++")]),a("td",[t._v("New Bridge")]),a("td",{staticClass:"add"},[t._v("2050")]),a("td",[t._v("Chimp N Zee")]),a("td",{staticClass:"remove"},[t._v("   ")])]),t._v(" "),a("tr",{staticClass:"remove"},[a("td",{staticClass:"desc"},[t._v("deleted row")]),a("td",[t._v("---")]),a("td",[t._v("Spamspan")]),a("td"),a("td",[t._v("S. Spamington")]),a("td",{staticClass:"remove"},[t._v("10000")])])])]),t._v(" "),a("p",[t._v("If the diff is on a spreadsheet table or CSV file, we’d generally want to respect row ordering. In this case, we can add context rows around insertions so we know where to put them. Less importantly, since they are going away anyway, we can do the same for deletions:")]),t._v(" "),a("div",{staticClass:"highlighter"},[a("table",[a("tr",{staticClass:"desc"},[a("td",{staticClass:"desc"}),a("td",[t._v("action"),a("br"),t._v("column")]),a("td",{attrs:{colspan:"4"}})]),t._v(" "),a("tr",{staticClass:"spec"},[a("td",{staticClass:"desc"},[t._v("schema row")]),a("td",[t._v("!")]),a("td"),a("td",{staticClass:"add"},[t._v("+++")]),a("td",[t._v("(designer)")]),a("td",{staticClass:"remove"},[t._v("---")])]),t._v(" "),a("tr",[a("td",{staticClass:"desc"},[t._v("header row")]),a("th",[t._v("@@")]),a("th",[t._v("bridge")]),a("th",{staticClass:"add"},[t._v("opened")]),a("th",[t._v("lead designer")]),a("th",{staticClass:"remove"},[t._v("length")])]),t._v(" "),a("tr",[a("td",{staticClass:"desc"},[t._v("omitted rows")]),a("td",[t._v("...")]),a("td",[t._v("...")]),a("td",[t._v("...")]),a("td",[t._v("...")]),a("td",[t._v("...")])]),t._v(" "),a("tr",[a("td",{staticClass:"desc"},[t._v("context row")]),a("td",[t._v("+")]),a("td",[t._v("Williamsburg")]),a("td",{staticClass:"add"},[t._v("1903")]),a("td",[t._v("L. L. Buck")]),a("td",{staticClass:"remove"},[t._v("1600")])]),t._v(" "),a("tr",{staticClass:"add"},[a("td",{staticClass:"desc"},[t._v("inserted row")]),a("td",[t._v("+++")]),a("td",[t._v("New Bridge")]),a("td",{staticClass:"add"},[t._v("2050")]),a("td",[t._v("Chimp N Zee")]),a("td",{staticClass:"remove"},[t._v("   ")])]),t._v(" "),a("tr",[a("td",{staticClass:"desc"},[t._v("context row")]),a("td",[t._v("+")]),a("td",[t._v("Queensborough")]),a("td",{staticClass:"add"},[t._v("1909")]),a("td",[t._v("Palmer & Hornbostel")]),a("td",{staticClass:"remove"},[t._v("1182")])]),t._v(" "),a("tr",[a("td",{staticClass:"desc"},[t._v("omitted rows")]),a("td",[t._v("...")]),a("td",[t._v("...")]),a("td",[t._v("...")]),a("td",[t._v("...")]),a("td",[t._v("...")])]),t._v(" "),a("tr",[a("td",{staticClass:"desc"},[t._v("context row")]),a("td",[t._v("+")]),a("td",[t._v("George Washington")]),a("td",{staticClass:"add"},[t._v("1931")]),a("td",[t._v("O. H. Ammann")]),a("td",{staticClass:"remove"},[t._v("3500")])]),t._v(" "),a("tr",{staticClass:"remove"},[a("td",{staticClass:"desc"},[t._v("deleted row")]),a("td",[t._v("---")]),a("td",[t._v("Spamspan")]),a("td"),a("td",[t._v("S. Spamington")]),a("td",{staticClass:"remove"},[t._v("10000")])])])]),t._v(" "),a("p",[t._v("The action column for a context row may contain a blank, or "),a("code",[t._v(":")]),t._v(", or "),a("code",[t._v("+")]),t._v(". The "),a("code",[t._v(":")]),t._v(" tag signifies the context row was moved (and its location is now as in the "),a("code",[t._v("REMOTE")]),t._v(" table). The "),a("code",[t._v("+")]),t._v(" signifies that there are cells added on that row.")]),t._v(" "),a("h2",{attrs:{id:"expressing-a-modified-cell-value"}},[a("a",{staticClass:"header-anchor",attrs:{href:"#expressing-a-modified-cell-value"}},[t._v("#")]),t._v(" Expressing a modified cell value")]),t._v(" "),a("p",[t._v("If a row contains a cell whose value is different in the compared tables, then that row should be shown in the diff, with a tag in the action column that ends in "),a("code",[t._v("->")]),t._v(". Then, the modified cell will be represented by converting the "),a("code",[t._v("LOCAL")]),t._v(" and "),a("code",[t._v("REMOTE")]),t._v(" values to text (we have yet to say how) and using the action tag as a separator. So for example here we change the last cell in a row from “Green” to “Blue”:")]),t._v(" "),a("div",{staticClass:"highlighter"},[a("table",[a("tr",[a("td",{staticClass:"modify"},[t._v("->")]),a("td",[t._v("Gnome")]),a("td",[t._v("Home and Garden")]),a("td",{staticClass:"modify"},[t._v("Green->Blue")])])])]),t._v(" "),a("p",[t._v("The tag must be preceded with as many extra "),a("code",[t._v("-")]),t._v(" characters as are needed to avoid collision with any character sequence on that row. So here is another row with exactly one cell changed:")]),t._v(" "),a("div",{staticClass:"highlighter"},[a("table",[a("tr",[a("td",{staticClass:"modify"},[t._v("--\x3e")]),a("td",[t._v("Console")]),a("td",[t._v("Toddlers -> Teenagers")]),a("td",{staticClass:"modify"},[t._v("White--\x3ePale")])])])]),t._v(" "),a("p",[t._v("When encoding a cell change as a string, we lose information about the type of the cell value. One distinction that may be important to retain is the difference between a NULL or empty cell, and the empty string. The tabular diff uses the following encoding:")]),t._v(" "),a("ul",[a("li",[t._v("A NULL value, if available, represents itself.")]),t._v(" "),a("li",[t._v("The encoded string "),a("code",[t._v("NULL")]),t._v(" represents a NULL value.")]),t._v(" "),a("li",[t._v("The encoded string "),a("code",[t._v("_NULL")]),t._v(" represents the string “NULL”.")]),t._v(" "),a("li",[t._v("The encoded string "),a("code",[t._v("__NULL")]),t._v(" represents the string “_NULL”.")]),t._v(" "),a("li",[t._v("…")])]),t._v(" "),a("p",[t._v("The goal is that the diff can be safely converted to and from CSV by existing tools without changing its meaning. To that end:")]),t._v(" "),a("ul",[a("li",[t._v("For matching (e.g. on context lines) blank cells in the diff (either the NULL value or an empty string) should be treated as ambiguous, and match "),a("em",[t._v("either")]),t._v(" of NULL or an empty string if an exact match is not available.")]),t._v(" "),a("li",[t._v("When using a diff as a patch, and inserting new cells, a blank cell in the diff (either the NULL value or an empty string) should be treated as ambiguous, and the “right” thing done given the column type. If either value could be inserted, then the blank string should be inserted (since the encoded string "),a("code",[t._v("NULL")]),t._v(" is available to specifically identify the NULL value).")])]),t._v(" "),a("p",[t._v("Note that if the diff is being expressed in a table that allows nested structure (e.g. a JSON representation), a list representation for modified cells might be used to avoid this issue. There is no specification for that at this time.")]),t._v(" "),a("h2",{attrs:{id:"expressing-a-moved-row"}},[a("a",{staticClass:"header-anchor",attrs:{href:"#expressing-a-moved-row"}},[t._v("#")]),t._v(" Expressing a moved row")]),t._v(" "),a("div",{staticClass:"custom-block tip"},[a("p",{staticClass:"custom-block-title"},[t._v("TIP")]),t._v(" "),a("p",[t._v("This can be ignored for tables for which row order is meaningless, e.g. in typical relational databases.")])]),t._v(" "),a("p",[t._v("A row that have been moved in a table for which row order is meaningful is marked with a "),a("code",[t._v(":")]),t._v(" tag in the action column and placed in the order it appears in the "),a("code",[t._v("REMOTE")]),t._v(" table.")]),t._v(" "),a("p",[t._v("To avoid burdening human readers with too much arcana, tags are "),a("em",[t._v("not")]),t._v(" combined when multiple kinds of actions apply to a row or column. So for example, a context row that was moved and had a cell added will "),a("em",[t._v("not")]),t._v(" be tagged as "),a("code",[t._v(":+")]),t._v(" or "),a("code",[t._v("+:")]),t._v(" or such-like, but rather by "),a("code",[t._v(":")]),t._v(". Cell addition can be determined from the schema row. These weak tags are included as aids for highlighting to express the most significant thing to know about a row.")]),t._v(" "),a("h2",{attrs:{id:"expressing-a-moved-column"}},[a("a",{staticClass:"header-anchor",attrs:{href:"#expressing-a-moved-column"}},[t._v("#")]),t._v(" Expressing a moved column")]),t._v(" "),a("div",{staticClass:"custom-block tip"},[a("p",{staticClass:"custom-block-title"},[t._v("TIP")]),t._v(" "),a("p",[t._v("This can be ignored for tables for which column order is meaningless.")])]),t._v(" "),a("p",[t._v("A column that have been moved in a table for which column order is meaningful is marked with a "),a("code",[t._v(":")]),t._v(" tag in the schema row and placed in the order it appears in the "),a("code",[t._v("REMOTE")]),t._v(" table.")]),t._v(" "),a("p",[t._v("If a diff that contains a "),a("code",[t._v(":")]),t._v(" tag is used to patch a table for which column order is not meaningful, that tag should simply be ignored.")]),t._v(" "),a("h2",{attrs:{id:"reference-action-column-tags"}},[a("a",{staticClass:"header-anchor",attrs:{href:"#reference-action-column-tags"}},[t._v("#")]),t._v(" Reference: action column tags")]),t._v(" "),a("table",[a("thead",[a("tr",[a("th",[t._v("Symbol")]),t._v(" "),a("th",[t._v("Meaning")])])]),t._v(" "),a("tbody",[a("tr",[a("td",[a("code",[t._v("@@")])]),t._v(" "),a("td",[t._v("The header row, giving column names.")])]),t._v(" "),a("tr",[a("td",[a("code",[t._v("!")])]),t._v(" "),a("td",[t._v("The schema row, given column differences.")])]),t._v(" "),a("tr",[a("td",[a("code",[t._v("+++")])]),t._v(" "),a("td",[t._v("An inserted row (present in "),a("code",[t._v("REMOTE")]),t._v(", not present in "),a("code",[t._v("LOCAL")]),t._v(").")])]),t._v(" "),a("tr",[a("td",[a("code",[t._v("---")])]),t._v(" "),a("td",[t._v("A deleted row (present in "),a("code",[t._v("LOCAL")]),t._v(", not present in "),a("code",[t._v("REMOTE")]),t._v(").")])]),t._v(" "),a("tr",[a("td",[a("code",[t._v("->")])]),t._v(" "),a("td",[t._v("A row with at least one cell modified cell. "),a("code",[t._v("--\x3e")]),t._v(", "),a("code",[t._v("---\x3e")]),t._v(", "),a("code",[t._v("----\x3e")]),t._v(" etc. have the same meaning.")])]),t._v(" "),a("tr",[a("td",[t._v("Blank")]),t._v(" "),a("td",[t._v("A blank string or NULL marks a row common to "),a("code",[t._v("LOCAL")]),t._v(" and "),a("code",[t._v("REMOTE")]),t._v(", given for context.")])]),t._v(" "),a("tr",[a("td",[a("code",[t._v("...")])]),t._v(" "),a("td",[t._v("Declares that rows common to "),a("code",[t._v("LOCAL")]),t._v(" and "),a("code",[t._v("REMOTE")]),t._v(" are being skipped.")])]),t._v(" "),a("tr",[a("td",[a("code",[t._v("+")])]),t._v(" "),a("td",[t._v("A row with at least one added cell.")])]),t._v(" "),a("tr",[a("td",[a("code",[t._v(":")])]),t._v(" "),a("td",[t._v("A reordered row.")])])])]),t._v(" "),a("h2",{attrs:{id:"reference-schema-row-tags"}},[a("a",{staticClass:"header-anchor",attrs:{href:"#reference-schema-row-tags"}},[t._v("#")]),t._v(" Reference: schema row tags")]),t._v(" "),a("table",[a("thead",[a("tr",[a("th",[t._v("Symbol")]),t._v(" "),a("th",[t._v("Meaning")])])]),t._v(" "),a("tbody",[a("tr",[a("td",[a("code",[t._v("+++")])]),t._v(" "),a("td",[t._v("An inserted column (present in "),a("code",[t._v("REMOTE")]),t._v(", not present in "),a("code",[t._v("LOCAL")]),t._v(").")])]),t._v(" "),a("tr",[a("td",[a("code",[t._v("---")])]),t._v(" "),a("td",[t._v("A deleted column (present in "),a("code",[t._v("LOCAL")]),t._v(", not present in "),a("code",[t._v("REMOTE")]),t._v(").")])]),t._v(" "),a("tr",[a("td",[a("code",[t._v("()")])]),t._v(" "),a("td",[t._v("A renamed column (the name in "),a("code",[t._v("LOCAL")]),t._v(" is given in parenthesis, and the name in "),a("code",[t._v("REMOTE")]),t._v(" will be in the header row).")])]),t._v(" "),a("tr",[a("td",[t._v("Blank")]),t._v(" "),a("td",[t._v("A blank string or NULL marks a column common to "),a("code",[t._v("LOCAL")]),t._v(" and "),a("code",[t._v("REMOTE")]),t._v(", given for context.")])]),t._v(" "),a("tr",[a("td",[a("code",[t._v("...")])]),t._v(" "),a("td",[t._v("Declares that columns common to "),a("code",[t._v("LOCAL")]),t._v(" and "),a("code",[t._v("REMOTE")]),t._v(" are being skipped.")])]),t._v(" "),a("tr",[a("td",[a("code",[t._v(":")])]),t._v(" "),a("td",[t._v("A reordered column.")])])])])],1)}),[],!1,null,null,null);e.default=d.exports}}]); \ No newline at end of file diff --git a/assets/js/app.6e255495.js b/assets/js/app.2cbe3852.js similarity index 94% rename from assets/js/app.6e255495.js rename to assets/js/app.2cbe3852.js index 8e9593e2..499f9b78 100644 --- a/assets/js/app.6e255495.js +++ b/assets/js/app.2cbe3852.js @@ -1,8 +1,8 @@ -(window.webpackJsonp=window.webpackJsonp||[]).push([[0],[]]);!function(t){function e(e){for(var r,a,s=e[0],c=e[1],u=e[2],f=0,p=[];f=n.length?{value:void 0,done:!0}:(t=r(n,o),e.index+=t.length,{value:t,done:!1})}))},function(t,e,n){var r=n(6),o=n(7),i=n(37);t.exports=r?function(t,e,n){return o.f(t,e,i(1,n))}:function(t,e,n){return t[e]=n,t}},function(t,e,n){var r=n(53),o=Math.min;t.exports=function(t){return t>0?o(r(t),9007199254740991):0}},function(t,e,n){var r=n(2),o=n(12),i=n(8),a=n(73),s=n(79),c=n(30),u=c.get,l=c.enforce,f=String(String).split("String");(t.exports=function(t,e,n,s){var c,u=!!s&&!!s.unsafe,p=!!s&&!!s.enumerable,d=!!s&&!!s.noTargetGet;"function"==typeof n&&("string"!=typeof e||i(n,"name")||o(n,"name",e),(c=l(n)).source||(c.source=f.join("string"==typeof e?e:""))),t!==r?(u?!d&&t[e]&&(p=!0):delete t[e],p?t[e]=n:o(t,e,n)):p?t[e]=n:a(e,n)})(Function.prototype,"toString",(function(){return"function"==typeof this&&u(this).source||s(this)}))},function(t,e,n){var r=n(2),o=n(128),i=n(104),a=n(12),s=n(3),c=s("iterator"),u=s("toStringTag"),l=i.values;for(var f in o){var p=r[f],d=p&&p.prototype;if(d){if(d[c]!==l)try{a(d,c,l)}catch(t){d[c]=l}if(d[u]||a(d,u,f),o[f])for(var v in i)if(d[v]!==i[v])try{a(d,v,i[v])}catch(t){d[v]=i[v]}}}},function(t,e){var n=Array.isArray;t.exports=n},function(t,e,n){var r=n(35),o=n(20);t.exports=function(t){return r(o(t))}},function(t,e,n){var r=n(138),o="object"==typeof self&&self&&self.Object===Object&&self,i=r||o||Function("return this")();t.exports=i},function(t,e,n){var r=n(106),o=n(2),i=function(t){return"function"==typeof t?t:void 0};t.exports=function(t,e){return arguments.length<2?i(r[t])||i(o[t]):r[t]&&r[t][e]||o[t]&&o[t][e]}},function(t,e){t.exports=function(t){if(null==t)throw TypeError("Can't call method on "+t);return t}},function(t,e){t.exports=!1},function(t,e,n){var r=n(6),o=n(80),i=n(37),a=n(17),s=n(36),c=n(8),u=n(105),l=Object.getOwnPropertyDescriptor;e.f=r?l:function(t,e){if(t=a(t),e=s(e,!0),u)try{return l(t,e)}catch(t){}if(c(t,e))return i(!o.f.call(t,e),t[e])}},function(t,e){var n={}.toString;t.exports=function(t){return n.call(t).slice(8,-1)}},function(t,e){t.exports=function(t){if("function"!=typeof t)throw TypeError(String(t)+" is not a function");return t}},function(t,e,n){var r=n(229),o=n(232);t.exports=function(t,e){var n=o(t,e);return r(n)?n:void 0}},function(t,e,n){"use strict";function r(t,e,n,r,o,i,a,s){var c,u="function"==typeof t?t.options:t;if(e&&(u.render=e,u.staticRenderFns=n,u._compiled=!0),r&&(u.functional=!0),i&&(u._scopeId="data-v-"+i),a?(c=function(t){(t=t||this.$vnode&&this.$vnode.ssrContext||this.parent&&this.parent.$vnode&&this.parent.$vnode.ssrContext)||"undefined"==typeof __VUE_SSR_CONTEXT__||(t=__VUE_SSR_CONTEXT__),o&&o.call(this,t),t&&t._registeredComponents&&t._registeredComponents.add(a)},u._ssrRegister=c):o&&(c=s?function(){o.call(this,(u.functional?this.parent:this).$root.$options.shadowRoot)}:o),c)if(u.functional){u._injectStyles=c;var l=u.render;u.render=function(t,e){return c.call(e),l(t,e)}}else{var f=u.beforeCreate;u.beforeCreate=f?[].concat(f,c):[c]}return{exports:t,options:u}}n.d(e,"a",(function(){return r}))},function(t,e){t.exports=function(t){return null!=t&&"object"==typeof t}},function(t,e,n){var r,o=n(5),i=n(184),a=n(78),s=n(40),c=n(110),u=n(74),l=n(54),f=l("IE_PROTO"),p=function(){},d=function(t){return" - - + +
Frictionless Standards
Edit this page (opens new window)
Last Updated: 2/17/2020, 12:56:48 PM
- + diff --git a/csv-dialect/index.html b/csv-dialect/index.html index 5aaea9c8..74eefeb8 100644 --- a/csv-dialect/index.html +++ b/csv-dialect/index.html @@ -8,8 +8,8 @@ - - + +
Frictionless Standards
- + diff --git a/data-package-identifier/index.html b/data-package-identifier/index.html index 5e522dae..83a61873 100644 --- a/data-package-identifier/index.html +++ b/data-package-identifier/index.html @@ -8,8 +8,8 @@ - - + +
Frictionless Standards
- + diff --git a/data-package/index.html b/data-package/index.html index ecbafbad..b4531a63 100644 --- a/data-package/index.html +++ b/data-package/index.html @@ -8,8 +8,8 @@ - - + + - + diff --git a/data-resource/index.html b/data-resource/index.html index 6055d638..127ce1ad 100644 --- a/data-resource/index.html +++ b/data-resource/index.html @@ -8,8 +8,8 @@ - - + +
Frictionless Standards
- + diff --git a/fiscal-data-package--budgets/index.html b/fiscal-data-package--budgets/index.html index f1c0f60c..8608e7b5 100644 --- a/fiscal-data-package--budgets/index.html +++ b/fiscal-data-package--budgets/index.html @@ -8,8 +8,8 @@ - - + +
Frictionless Standards

WARNING

This is a draft specification and still under development. If you have comments or suggestions please file them in the issue tracker (opens new window). If you have explicit changes please fork the git repo (opens new window) and submit a pull request.

# Fiscal Data Package - Budget Standard Taxonomy

The Budget Taxonomy is a set of ColumnTypes to be used in the context of a Fiscal Data Package to describe budget data of organizations (governments or otherwise.)

Author(s)
Created 14 March 2014
Updated 22 April 2018
JSON Schema fiscal-data-package-budgets.json
Version 1.0-rc.1

# Language

The key words MUST, MUST NOT, REQUIRED, SHALL, SHALL NOT, SHOULD, SHOULD NOT, RECOMMENDED, MAY, and OPTIONAL in this document are to be interpreted as described in RFC 2119

# Changelog

  • 1.0.0rc1: Initial text

# Introduction

This document contains a ColumnType taxonomy to be used for publishing budget data files.

The ColumnTypes contained in this taxonomy contain:

  • Generic value types
  • Generic time types as well as the more specific ‘fiscal year’ type
  • Classifications:
    • Functional: COFOG and Generic
    • Economic: GFSM and Generic
    • Administrative
    • Activity
  • Other Budgeting-related Properties
  • Geo-Related types

# References

# Location

The canonic location for this taxonomy’s ColumnType definition - to be used in fiscal data package descriptors - is

https://specs.frictionlessdata.io/taxonomies/fiscal/budgets.json

# The Taxonomy

# Amounts and their properties

# value

Numeric value depicting a fiscal amount related to the budget item, spending transaction etc.

  • dataType: number
# value-kind:code

Unique identifier for the amount kind

  • dataType: string
  • unique: True
# value-kind:label

Display name for the amount kind

  • dataType: string
  • labelOf: value-kind:code
# value-currency:code

Unique identifier for the amount currency

  • dataType: string
  • unique: True
# value-currency:label

Display name for the amount currency

  • dataType: string
  • labelOf: value-kind:code

# Time Indication

# date:fiscal-year

The fiscal-year for which the values in this record are relevant

  • dataType: integer
  • unique: True
# date:fiscal:activity-approval

The approval date of a specific activity

  • dataType: date
  • unique: True
# date:fiscal:activity-end

The ending date of a specific activity

  • dataType: date
  • unique: True
# date:fiscal:activity-start

The starting date of a specific activity

  • dataType: date
  • unique: True
# date:fiscal:final-payment

The date of the last payment for a specific activity

  • dataType: date
  • unique: True
# date:fiscal:first-payment

The date of the first payment for a specific activity

  • dataType: date
  • unique: True
# date:generic

An non-specific date related to the values in this record (e.g. transaction date etc.)

  • dataType: date
  • unique: True

# Classifications: Functional (COFOG)

# functional-classification:cofog:class:code

The COFOG ‘Class’ Level code

  • dataType: string
  • prior: functional-classification:cofog:group:code
  • unique: True
# functional-classification:cofog:class:description

A more detailed textual description for this class

  • dataType: string
# functional-classification:cofog:class:label

A label or display name for this class

  • dataType: string
  • labelOf: functional-classification:cofog:class:code
# functional-classification:cofog:code

The complete COFOG classification code, non level-specific

  • dataType: string
  • unique: True
# functional-classification:cofog:description

Description for this COFOG classification, non level-specific

  • dataType: string
# functional-classification:cofog:division:code

The COFOG ‘Division’ Level code

  • dataType: string
  • unique: True
# functional-classification:cofog:division:description

A more detailed textual description for this division

  • dataType: string
# functional-classification:cofog:division:label

A label or display name for this division

  • dataType: string
  • labelOf: functional-classification:cofog:division:code
# functional-classification:cofog:group:code

The COFOG ‘Group’ Level code

  • dataType: string
  • prior: functional-classification:cofog:division:code
  • unique: True
# functional-classification:cofog:group:description

A more detailed textual description for this group

  • dataType: string
# functional-classification:cofog:group:label

A label or display name for this group

  • dataType: string
  • labelOf: functional-classification:cofog:group:code
# functional-classification:cofog:label

Display name for this COFOG classification, non level-specific

  • dataType: string
  • labelOf: functional-classification:cofog:code

# Classifications: Functional (non-specific)

# functional-classification:generic:code

A code or unique identifier for the classification (not level specific)

  • dataType: string
  • unique: True
# functional-classification:generic:description

A longer descriptive text for this classification

  • dataType: string
# functional-classification:generic:label

A label, title or display name for the classification

  • dataType: string
  • labelOf: functional-classification:generic:code
# functional-classification:generic:level{1..8}:code

A code or unique identifier for the top level of the classification

  • dataType: string
  • unique: True
# functional-classification:generic:level{1..8}:description

A longer descriptive text for this level of the classification

  • dataType: string
# functional-classification:generic:level{1..8}:label

A label, title or display name for this level of the classification

  • dataType: string
  • labelOf: functional-classification:generic:level1:code

# Classifications: Economic (GFSM)

# economic-classification:gfsm:level{1..4}:code

A code or unique identifier for the top level of the classification

  • dataType: string
  • unique: True
# economic-classification:gfsm:level{1..4}:description

A longer descriptive text for this level of the classification

  • dataType: string
# economic-classification:gfsm:level{1..4}:label

A label, title or display name for this level of the classification

  • dataType: string
  • labelOf: economic-classification:gfsm:level1:code

# Classifications: Economic (non-specific)

# economic-classification:generic:code

A code or unique identifier for the classification (not level specific)

  • dataType: string
  • unique: True
# economic-classification:generic:description

A longer descriptive text for this classification

  • dataType: string
# economic-classification:generic:label

A label, title or display name for the classification

  • dataType: string
  • labelOf: economic-classification:generic:code
# economic-classification:generic:level{1..4}:code

A code or unique identifier for the top level of the classification

  • dataType: string
  • unique: True
# economic-classification:generic:level{1..4}:description

A longer descriptive text for this level of the classification

  • dataType: string
# economic-classification:generic:level{1..4}:label

A label, title or display name for this level of the classification

  • dataType: string
  • labelOf: economic-classification:generic:level1:code

# Classifications: Administrative

# administrative-classification:generic:code

A code or unique identifier for the classification (not level specific)

  • dataType: string
  • unique: True
# administrative-classification:generic:description

A longer descriptive text for this classification

  • dataType: string
# administrative-classification:generic:label

A label, title or display name for the classification

  • dataType: string
  • labelOf: administrative-classification:generic:code
# administrative-classification:generic:level{1..8}:code

A code or unique identifier for the top level of the classification

  • dataType: string
  • unique: True
# administrative-classification:generic:level{1..8}:description

A longer descriptive text for this level of the classification

  • dataType: string
# administrative-classification:generic:level{1..8}:label

A label, title or display name for this level of the classification

  • dataType: string
  • labelOf: administrative-classification:generic:level1:code
# activity:generic:contract:code

A code or unique identifier for the contract

  • dataType: string
  • prior: activity:generic:subproject:code
  • unique: True
# activity:generic:contract:label

A label, title or display name for this contract

  • dataType: string
  • labelOf: activity:generic:contract:code
# activity:generic:program:code

A code or unique identifier for the program

  • dataType: string
  • unique: True
# activity:generic:program:label

A label, title or display name for this program

  • dataType: string
  • labelOf: activity:generic:program:code
# activity:generic:project:code

A code or unique identifier for the project

  • dataType: string
  • prior: activity:generic:subprogram:code
  • unique: True
# activity:generic:project:label

A label, title or display name for this project

  • dataType: string
  • labelOf: activity:generic:project:code
# activity:generic:subprogram:code

A code or unique identifier for the subprogram

  • dataType: string
  • prior: activity:generic:program:code
  • unique: True
# activity:generic:subprogram:label

A label, title or display name for this subprogram

  • dataType: string
  • labelOf: activity:generic:subprogram:code
# activity:generic:subproject:code

A code or unique identifier for the sub-project

  • dataType: string
  • prior: activity:generic:project:code
  • unique: True
# activity:generic:subproject:label

A label, title or display name for this sub-project

  • dataType: string
  • labelOf: activity:generic:subproject:code
# budget-line-id

A unique identifier for this budget line

  • dataType: string
  • unique: True
# budgetary-transfers

Extra properties regarding whether the expenditure contains budgetary transfers

  • dataType: string
# direction

Specifies whether the values in this line are expenditure or revenues

  • dataType: string
  • unique: True
# phase:id

The phase identifier

  • dataType: string
  • unique: True
# phase:label

The phase display name

  • dataType: string
  • labelOf: phase:id
# expenditure-type:code

Unique identifier for the expenditure type

  • dataType: string
  • unique: True
# expenditure-type:label

Display name for the expenditure type

  • dataType: string
  • labelOf: expenditure-type:code
# fin-source:generic:code

A code or unique identifier for the financial source

  • dataType: string
  • unique: True
# fin-source:generic:label

Display name or title for the financial source

  • dataType: string
  • labelOf: fin-source:generic:code
# fin-source:generic:level{1..3}:code

A code or unique identifier for the top level of the financial source hierarchy

  • dataType: string
  • unique: True
# fin-source:generic:level{1..3}:label

Display name or title for the top level of the financial source hierarchy

  • dataType: string
  • labelOf: fin-source:generic:level1:code

# Geographic Information

# geo:generic:code

Unique identifier or code for Geographic feature specified in the data

  • dataType: string
  • unique: True
# geo:generic:codeList

Indicates a specific code list from which a Geographic identifier is drawn

  • dataType: string
  • unique: True
# geo:generic:title

The display name of the geographic feature

  • dataType: string
  • labelOf: geo:generic:code
# geo:source:code

Unique identifier or code for Geographic feature specified in the data

  • dataType: string
  • unique: True
# geo:source:codeList

Indicates a specific code list from which a Geographic identifier is drawn

  • dataType: string
  • unique: True
# geo:source:title

The display name of the geographic feature

  • dataType: string
  • labelOf: geo:source:code
# geo:target:code

Unique identifier or code for Geographic feature specified in the data

  • dataType: string
  • unique: True
# geo:target:codeList

Indicates a specific code list from which a Geographic identifier is drawn

  • dataType: string
  • unique: True
# geo:target:level{1..2}:code

Unique identifier or code for the Top Level Geographic feature specified in the data

  • dataType: string
  • unique: True
# geo:target:level{1..2}:title

The display name for the Top Level Geographic feature specified in the data

  • dataType: string
  • labelOf: geo:target:level1:code
# geo:target:title

The display name of the geographic feature

  • dataType: string
  • labelOf: geo:target:code
- + diff --git a/fiscal-data-package--spending/index.html b/fiscal-data-package--spending/index.html index dfeb0dc9..f5d0d1a7 100644 --- a/fiscal-data-package--spending/index.html +++ b/fiscal-data-package--spending/index.html @@ -8,8 +8,8 @@ - - + +
Frictionless Standards

WARNING

This is a draft specification and still under development. If you have comments or suggestions please file them in the issue tracker (opens new window). If you have explicit changes please fork the git repo (opens new window) and submit a pull request.

# Fiscal Data Package - Spending Standard Taxonomy

The Budget Taxonomy is a set of ColumnTypes to be used in the context of a Fiscal Data Package to describe spending data of organizations (governments or otherwise.)

Author(s)
Created 14 March 2014
Updated 22 April 2018
JSON Schema fiscal-data-package-spending.json
Version 1.0-rc.1

# Language

The key words MUST, MUST NOT, REQUIRED, SHALL, SHALL NOT, SHOULD, SHOULD NOT, RECOMMENDED, MAY, and OPTIONAL in this document are to be interpreted as described in RFC 2119

# Changelog

  • 1.0.0rc1: Initial text

# Introduction

This document contains a ColumnType taxonomy to be used for publishing spending data files. It should be used in conjunction with the budget taxonomy, as it contains some common ColumnTypes as well.

The ColumnTypes contained in this taxonomy contain:

  • Transactions Identifiers
  • Details about administrators, procurers, suppliers and recipients
  • Some Geographic related types (esp. for addresses)

# References

# Location

The canonic location for this taxonomy’s ColumnType definition - to be used in fiscal data package descriptors - is

https://specs.frictionlessdata.io/taxonomies/fiscal/spending.json

# The Taxonomy

# Amounts and their properties

# Geographic Information

# geo:address:city:code

The code of the city part of the address

  • dataType: string
  • prior: geo:address:county:code
  • unique: True
# geo:address:city:label

The name of the city part of the address

  • dataType: string
  • labelOf: geo:address:city:code
# geo:address:country:code

The code of the country part of the address

  • dataType: string
  • unique: True
# geo:address:country:label

The name of the country part of the address

  • dataType: string
  • labelOf: geo:address:country:code
# geo:address:county:code

The code of the county part of the address

  • dataType: string
  • prior: geo:address:region:code
  • unique: True
# geo:address:county:label

The name of the county part of the address

  • dataType: string
  • labelOf: geo:address:county:code
# geo:address:region:code

The code of the region part of the address

  • dataType: string
  • prior: geo:address:country:code
  • unique: True
# geo:address:region:label

The name of the region part of the address

  • dataType: string
  • labelOf: geo:address:region:code
# geo:address:street-address:description

Actual street address in whole address

  • dataType: string
# geo:address:zip:code

The postal code in the address

  • dataType: string
  • prior: geo:address:city:code
  • unique: True

# Actors involved in the Transaction (Administrator, Procurer)

# administrator:generic:id

Unique identifier for the Administrator

  • dataType: string
  • unique: True
# administrator:generic:name

The display name for the Administrator

  • dataType: string
  • labelOf: administrator:generic:id
# procurer:bank:account

Unique identifier for the bank account of the Procurer

  • dataType: string
# procurer:bank:branch:code

Unique identifier of the bank’s branch of the Procurer

  • dataType: string
  • unique: True
# procurer:bank:branch:name

Name of the bank’s branch of the Procurer

  • dataType: string
  • labelOf: procurer:bank:branch:code
# procurer:bank:code

Unique identifier for the bank of the Procurer

  • dataType: string
  • unique: True
# procurer:generic:id

Unique identifier for the Procurer

  • dataType: string
  • unique: True
# procurer:generic:name

The display name of the Procurer

  • dataType: string
  • labelOf: procurer:generic:id

# Recipient of the Transaction

# recipient:bank:account

Unique identifier for the bank account of the Recipient

  • dataType: string
# recipient:bank:branch:code

Unique identifier of the bank’s branch of the Recipient

  • dataType: string
  • unique: True
# recipient:bank:branch:name

Name of the bank’s branch of the Recipient

  • dataType: string
  • labelOf: recipient:bank:branch:name
# recipient:bank:code

Unique identifier for the bank of the Recipient

  • dataType: string
  • unique: True
# recipient:generic:id

Unique identifier for the Recipient

  • dataType: string
  • unique: True

Unique identifier for the codelist from which the legal entity code comes from

  • dataType: string
  • prior: recipient:generic:id
  • unique: True

Unique identifier for the legal entity

  • dataType: string
  • prior: recipient:generic:legal-entity:code-type
  • unique: True

Trading name (or other) of the legal entity

  • dataType: string
  • labelOf: recipient:generic:legal-entity:code

Text describing the representative of the legal entity

  • dataType: string

Code of the specific project inside the legal entity

  • dataType: string
  • prior: recipient:generic:legal-entity:code
  • unique: True

Name of the specific project inside the legal entity

  • dataType: string

Name of the specific project inside the legal entity

  • dataType: string
  • labelOf: recipient:generic:legal-entity:receiving-project:code

Status of the specific project inside the legal entity

  • dataType: string
# recipient:generic:name

The display name for the Recipient

  • dataType: string
  • labelOf: recipient:generic:id
# recipient:generic:url

An Internet address for the Recipient

  • dataType: string

# Supplier Details

# supplier:generic:id

Unique identifier for the Supplier

  • dataType: string
  • unique: True
# supplier:generic:name

The display name for the Supplier

  • dataType: string
  • labelOf: supplier:generic:id

# Transaction Details

# transaction-id:budget-code

Unique identifier for the Budget Line for this transaction

  • dataType: string
  • unique: True
# transaction-id:code

A Unique identifier for this transaction

  • dataType: string
  • unique: True
# transaction-id:contract-id

Unique identifier for the Contract for this transaction

  • dataType: string
  • unique: True
# transaction-id:court-order

Unique identifier for the Court Order for this transaction

  • dataType: string
  • unique: True
# transaction-id:invoice-id

Unique identifier for the Invoice for this transaction

  • dataType: string
  • unique: True
# transaction-id:purchase-order

Unique identifier for the Purchase Order for this transaction

  • dataType: string
  • unique: True
# transaction-id:tender-id

Unique identifier for the Tender for this transaction

  • dataType: string
  • unique: True
# transaction-id:tender-kind

Unique identifier for the Tender Kind for this transaction

  • dataType: string
# transaction-id:transaction-kind

Unique identifier for the Transaction Kind for this transaction

  • dataType: string
- + diff --git a/fiscal-data-package/index.html b/fiscal-data-package/index.html index 8ab63cb9..0753767e 100644 --- a/fiscal-data-package/index.html +++ b/fiscal-data-package/index.html @@ -8,8 +8,8 @@ - - + +
Frictionless Standards
Edit this page (opens new window)
Last Updated: 5/28/2020, 10:00:53 AM
- + diff --git a/guides/data-package/index.html b/guides/data-package/index.html index 9a0fa535..31654a11 100644 --- a/guides/data-package/index.html +++ b/guides/data-package/index.html @@ -8,8 +8,8 @@ - - + +
Frictionless Standards
- + diff --git a/guides/implementation/index.html b/guides/implementation/index.html index c13df12a..8019bcf6 100644 --- a/guides/implementation/index.html +++ b/guides/implementation/index.html @@ -8,8 +8,8 @@ - - + +
Frictionless Standards
Back to the main site (opens new window)

# Implementation

This document is meant to serve as an introduction and an entry point into writing a library that implements a Frictionless Data specification. The focus is on two libraries in particular - Data Package and Table Schema - as implementing these libraries essentially implements the whole family of specifications.

The reader, being an implementer/maintainer of such libraries, should get a clear understanding of the reference material available for undertaking work, and the minimal set of actions that such libraries must enable for their users.

We prefer to focus on actions rather than features, feature sets, user stories, or more formal API specifications as we want to leave enough flexibility for implementations that follow the idioms of the host language, yet we do want to ensure a common base of what can be done with an implementation in any language.

# Context

While OKI and various other 3rd parties have been using the Data Package family of specifications with great success for several years, it has mostly been over the last 12 months that we are starting to see more mature libraries to implement the specifications at a “low level” for ease of reuse.

At present, we consider the libraries maintained by Open Knowledge International in both Python and JavaScript to be reference implementations that serve as a guide for how to approach the specifications in code, and the type of actions that are enabled for users of the libraries. Further, we make extensive use of JSON Schema (opens new window) to validate descriptors that are passed to these libraries. This enables significant reuse across implementations for descriptor validation logic.

# High-level requirements

Here we will describe the minimal requirements for implementing Frictionless Data specifications in a given programming language. Based on the Python and JavaScript implementations, the implementations are split across two packages: Data Package and Table Schema.

Also, see the stack reference (opens new window) section below for some naming conventions we use, and that ideally should be followed in new implementations.

# Data Package library

The Data Package library can load and validate any descriptor for a Data Package Profile, allow the creation and modification of descriptors, and expose methods for reading and streaming data in the package. When a descriptor is a Tabular Data Package, it uses the Table Schema library, and exposes its functionality, for each resource object in the resources array.

# References

# Actions

  • read an existing Data Package descriptor
  • validate an existing Data Package descriptor, including profile-specific validation via the registry of JSON Schemas
  • create a new Data Package descriptor
  • edit an existing Data Package descriptor
  • as part of editing a descriptor, helper methods to add and remove resources from the resources array
  • validate edits made to a data package descriptor
  • save a Data Package descriptor to a file path
  • zip a Data Package descriptor and its co-located references (more generically: “zip a data package”)
  • read a zip file that “claims” to be a data package
  • save a zipped Data Package to disk

# Examples

# Table Schema library

The Table Schema library can load and validate any Table Schema descriptor, allow the creation and modification of descriptors, expose methods for reading and streaming data that conforms to a Table Schema via the Tabular Data Resource abstraction.

# References

# Actions

  • read an existing Table Schema descriptor
  • validate an existing Table Schema descriptor using the JSON Schema spec
  • create a new Table Schema descriptor
  • edit an existing Table Schema descriptor
  • provide a model-type interface to interact with a descriptor
  • infer a Table Schema descriptor from a supplied sample of data
  • validate a data source against the Table Schema descriptor, including in response to editing the descriptor
  • enable streaming and reading of a data source through a Table Schema (cast on iteration)

# Examples

# On dereferencing and descriptor validation

Some properties in the Frictionless Data specifications allow a path (a URL or a POSIX path) that resolves to an object.

The most prominent example of this is the schema property on Tabular Data Resource descriptors.

Allowing such references has practical use for publishers, for example in allowing schema reuse. However, it does introduce difficulties in the validation of such properties. For example, validating a path pointing to a schema rather than the schema object itself will do little to guarantee the integrity of the schema definition. Therefore implementors MUST dereference such “referential” property values before attempting to validate a descriptor. At present, this requirement applies to the following properties in Tabular Data Package and Tabular Data Resource:

  • schema
  • dialect

# Other libraries

Data Package and Table Schema implement the core Frictionless Data specifications. The JavaScript implementations maintained by Open Knowledge International essentially follow the above requirements as is. However, our Python toolchain has some additional libraries - goodtables and tabulator - which are an important part of the Frictionless Data stack, and we would be delighted to see them implemented in other languages either as standalone libraries, or, as part of a wider effort in implementing Data Package and Table Schema.

# tabulator

tabulator provides a consistent interface for streaming reading and writing of tabular data. It supports CSV, which is required for Table Schema, Tabular Data Resource, and Tabular Data Package, and also supports Excel, JSON, newline delimited JSON, Google Sheets, and ODS.

# References

# goodtables

goodtables validates tabular data, checking for structural and schematic errors, and producing reports that can be used to iterate on data file sources as part of common data publication work flows. goodtables uses tabulator, tableschema, and datapackage internally, as well as implementing data-quality-spec.

It may be of general interest that goodtables is also available as a service - goodtables.io (opens new window) - providing continuous data validation in the style of CI solutions for code.

# References

# Work process

Open Knowledge International coding standards can be found here (opens new window). While some of the standards document refers to idioms in JavaScript and Python, much of it is about more general standards around using git, testing requirements and so on. These need to be adhered to.

We have some small example code repositories in Python and JavaScript that demonstrate these coding standards.

If you would like to contribute sections based on idioms in your target language, that would be great: it will serve as a further reference to others, and also have the added benefit of enabling our team to learn from you.

- + diff --git a/index.html b/index.html index 3597b795..9ec946ae 100644 --- a/index.html +++ b/index.html @@ -8,8 +8,8 @@ - - + +
Frictionless Standards
- + diff --git a/patterns/index.html b/patterns/index.html index fa23fd23..353902c8 100644 --- a/patterns/index.html +++ b/patterns/index.html @@ -8,8 +8,8 @@ - - + +
Frictionless Standards
- + diff --git a/profiles/index.html b/profiles/index.html index 46d71143..e2c3c525 100644 --- a/profiles/index.html +++ b/profiles/index.html @@ -8,8 +8,8 @@ - - + +
Frictionless Standards

# Profiles

Data Package and Data Resource Profiles

Author(s) Paul Walsh, Rufus Pollock
Created 11 December 2016
Updated 24 May 2017
JSON Schema profiles.json
Version

# Language

The key words MUST, MUST NOT, REQUIRED, SHALL, SHALL NOT, SHOULD, SHOULD NOT, RECOMMENDED, MAY, and OPTIONAL in this document are to be interpreted as described in RFC 2119

# Introduction

Different kinds of data need different data and metadata formats. To support these different data and metadata formats we need to extend and specialise the generic Data Package. These specialized types of Data Package (or Data Resource) are termed profiles. For example, there is a Tabular Data Package profile that specializes Data Packages specifically for tabular data.

Thus, every Package and Resource descriptor has a profile. The namespace for the profile is the type of descriptor, so the default profile, if none is declared, is data-package for a Package descriptor and data-resource for a Resource descriptor.

In summary, an extension of Data Package may be formalised as a profile. A profile is a Data Package which extends the default specification towards more specific needs.

# profile Property

In addition to the concept, we need an explicit way for publishers to state the profile they are using and consumers to discover this.

Thus, we have a profile property that declares the profile for the descriptor for this Package or Resource. For the default Data Package and Data Resource descriptor, this SHOULD be present with a value of data-package/data-resource, but if not, the absence of a profile is equivalent to setting "profile": "data-package"/ "profile": "data-resource".

Custom profiles MUST have a profile property, where the value is a unique identifier for that profile. This unique identifier MUST be a string and can be in one of two forms. It can be an id from the official Data Package Schema Registry, or, a fully-qualified URL that points directly to a JSON Schema that can be used to validate the profile.

As part of the Frictionless Data Specifications project, we publish a number of Data Package profiles such as:

We also publish the following Data Resource profiles:

- + diff --git a/security/index.html b/security/index.html index e9bead43..241f3dd7 100644 --- a/security/index.html +++ b/security/index.html @@ -8,8 +8,8 @@ - - + +
Frictionless Standards
- + diff --git a/table-schema/index.html b/table-schema/index.html index 78a8a4c6..26728cc5 100644 --- a/table-schema/index.html +++ b/table-schema/index.html @@ -8,8 +8,8 @@ - - + + - + diff --git a/tabular-data-package/index.html b/tabular-data-package/index.html index 4e67f50e..98de8248 100644 --- a/tabular-data-package/index.html +++ b/tabular-data-package/index.html @@ -8,8 +8,8 @@ - - + +
Frictionless Standards
- + diff --git a/tabular-data-resource/index.html b/tabular-data-resource/index.html index 8fb0d821..dbee953c 100644 --- a/tabular-data-resource/index.html +++ b/tabular-data-resource/index.html @@ -8,8 +8,8 @@ - - + + - + diff --git a/tabular-diff/index.html b/tabular-diff/index.html index 9a886b93..1a5e2959 100644 --- a/tabular-diff/index.html +++ b/tabular-diff/index.html @@ -8,8 +8,8 @@ - - + +
Frictionless Standards

# Tabular Diff Format

The Tabular Diff Format is a format for expressing the difference between two tables such that the difference is itself in tabular form.

Author(s) Paul Fitzpatrick
Created 16 December 2011
Updated 27 May 2014
JSON Schema
Version 0.8.0

# Language

The key words MUST, MUST NOT, REQUIRED, SHALL, SHALL NOT, SHOULD, SHOULD NOT, RECOMMENDED, MAY, and OPTIONAL in this document are to be interpreted as described in RFC 2119

# Summary

The tabular diff format expresses the difference between two versions of a table.
Here’s an example of a diff:

@@bridgedesignerlength
BrooklynJ. A. Roebling1595
+++ManhattanG. Lindenthal1470
->WilliamsburgD. Duck->L. L. Buck1600
QueensboroughPalmer & Hornbostel1182
............
George WashingtonO. H. Ammann3500
---SpamspanS. Spamington10000

As for text diffs, the format emphasizes significant changes. Also like text diffs, the format is unambiguous without color, but readily enhanced with it. Unlike text diffs, the format preserves the original tabular structure, allowing presentation with sensible visual alignment.

There is a reference implementation of a tool for generating and processing tabular diffs at https://github.com/paulfitz/daff (opens new window).

# Scope

The tabular diff format can express the following kinds of changes in a table:

  • Inserted or deleted rows.
  • Inserted, deleted, or renamed columns.
  • Modified cell values.

If the order of the rows or columns of the the table are meaningful, then the format can also express:

  • Reordered rows or columns.

Changes in formatting and systematic transformation of data (such as capitalization) are not expressible.

# General structure

Assume we have two tables, called LOCAL and REMOTE. The diff summarizes the changes needed to modify LOCAL to match REMOTE.

The diff contains rows and columns from the tables being compared. As in regular text diffs, there is flexibility in what data is given and what is omitted.

  • A column or row that is common to the tables being compared should appear at most once.
  • Any column or row containing a modified cell should be included in the diff, and the modified cell should be represented using the procedure in Expressing a modified cell value.
  • Columns or rows that are present in one table and not in the other should be included in the diff.
  • Columns or rows that are unchanged and unneeded for context may be omitted, at the diff creator’s discretion.
  • Omitted blocks of rows or columns should be marked with a row/column full of ... cells.

In addition, the diff contains the following special rows and columns:

  • The action column. This is always present, and is the first column in the diff if columns are ordered. If columns are not ordered, it is the column named __hilite_diff__.
  • A header row with column names. This row can be recognized since it will have the tag @@ in the action column.
  • A schema row that is needed when the column structure differs between tables. This row can be recognized since it will have the tag ! in the action column.

Here’s an example diff, where the tables being compared share the same three columns:

action columndata from compared tables
header row@@bridgedesignerlength
BrooklynJ. A. Roebling1595
+++ManhattanG. Lindenthal1470
->WilliamsburgD. Duck->L. L. Buck1600
QueensboroughPalmer & Hornbostel1182
omitted rows............
George WashingtonO. H. Ammann3500
---SpamspanS. Spamington10000

The colors do not make up part of this specification, they are just syntax highlighting. The meaning of the various tags will become clear in later sections, for now we are just concerned with the structure of the diff. Here’s an example where there is a difference in columns: LOCAL has a length column that is removed in REMOTE, REMOTE has an opened column that wasn’t present in LOCAL, and the designer column in LOCAL is renamed as lead designer in REMOTE:

action columndata from compared tables
schema row!+++(designer)---
header row@@bridgeopenedlead designerlength
+Brooklyn1883J. A. Roebling1595
+Manhattan1909G. Lindenthal1470
+Williamsburg1903L. L. Buck1600
+Queensborough1909Palmer & Hornbostel1182
+Triborough1936O. H. Ammann1380,383
+Bronx Whitestone1939O. H. Ammann2300
+Throgs Neck1961O. H. Ammann1800
+George Washington1931O. H. Ammann3500

We see that a schema row is added above the header row to represent the changes in columns. With this general anatomy of a diff in mind, let’s look at the details of how to interpret it.

TIP

If writing a rule to “sniff” a file to see if it is a tabular diff, the @@ tag is a handy tell-tale. But watch out for that schema row! Also, to allow for future evolution of this format, please try to be robust to a few extra rows or columns appearing before the @@.

# Expressing inserted and deleted columns

An inserted column is expressed simply by including that column in the diff, and placing +++ in the schema row above the corresponding column name in the header row. Similarly, a deleted column is expressed by including that column in the diff, and placing --- in the schema row above the corresponding column name. As a special case, a renamed column is represented by simply placing its old name in parentheses in the schema row.

In our earlier example, LOCAL has the columns bridge, designer, and length, while REMOTE has the columns bridge, opened, and lead designer (designer renamed). So opened is inserted and length is deleted:

action
column		data from compared tables
schema row!+++(designer)---
header row@@bridgeopenedlead designerlength
+Brooklyn1883J. A. Roebling1595
+Manhattan1909G. Lindenthal1470
+Williamsburg1903L. L. Buck1600

If we are dealing with a data store where columns are unordered, then likewise column order in the diff is irrelevant. Otherwise, the inserted and deleted rows should be placed in their appropriate order.

Any rows in the diff that are present only the LOCAL table will leave inserted columns blank. Similarly, any rows in the diff that are present only in the REMOTE table will leave deleted columns blank. Rows that are present in both tables will have values in all cells.

# Expressing inserted and deleted rows

An inserted row is expressed simply by placing +++ in the action column, and placing cell values in the appropriate columns. If there are columns in the diff that are in LOCAL but not in REMOTE, these are left blank. Likewise, a deleted row is expressed by placing --- in the action column, and its cell values in the appropriate columns. If there are columns in the diff that are in REMOTE but not in LOCAL, these are left blank. For example, suppose in REMOTE there is a row about a New Bridge that wasn’t in LOCAL, and a row about a bridge called Spamspan has been dropped. Here is what the inserted and deleted rows would look like, lined up with the header row for reference:

action
column
schema row!+++(designer)---
header row@@bridgeopenedlead designerlength
 
inserted row+++New Bridge2050Chimp N Zee   
 
deleted row---SpamspanS. Spamington10000

If the diff is on a database table where rows have no ordering, then we can just stick these rows together and we have our diff:

action
column
schema row!+++(designer)---
header row@@bridgeopenedlead designerlength
inserted row+++New Bridge2050Chimp N Zee   
deleted row---SpamspanS. Spamington10000

If the diff is on a spreadsheet table or CSV file, we’d generally want to respect row ordering. In this case, we can add context rows around insertions so we know where to put them. Less importantly, since they are going away anyway, we can do the same for deletions:

action
column
schema row!+++(designer)---
header row@@bridgeopenedlead designerlength
omitted rows...............
context row+Williamsburg1903L. L. Buck1600
inserted row+++New Bridge2050Chimp N Zee   
context row+Queensborough1909Palmer & Hornbostel1182
omitted rows...............
context row+George Washington1931O. H. Ammann3500
deleted row---SpamspanS. Spamington10000

The action column for a context row may contain a blank, or :, or +. The : tag signifies the context row was moved (and its location is now as in the REMOTE table). The + signifies that there are cells added on that row.

# Expressing a modified cell value

If a row contains a cell whose value is different in the compared tables, then that row should be shown in the diff, with a tag in the action column that ends in ->. Then, the modified cell will be represented by converting the LOCAL and REMOTE values to text (we have yet to say how) and using the action tag as a separator. So for example here we change the last cell in a row from “Green” to “Blue”:

->GnomeHome and GardenGreen->Blue

The tag must be preceded with as many extra - characters as are needed to avoid collision with any character sequence on that row. So here is another row with exactly one cell changed:

-->ConsoleToddlers -> TeenagersWhite-->Pale

When encoding a cell change as a string, we lose information about the type of the cell value. One distinction that may be important to retain is the difference between a NULL or empty cell, and the empty string. The tabular diff uses the following encoding:

  • A NULL value, if available, represents itself.
  • The encoded string NULL represents a NULL value.
  • The encoded string _NULL represents the string “NULL”.
  • The encoded string __NULL represents the string “_NULL”.
  • …

The goal is that the diff can be safely converted to and from CSV by existing tools without changing its meaning. To that end:

  • For matching (e.g. on context lines) blank cells in the diff (either the NULL value or an empty string) should be treated as ambiguous, and match either of NULL or an empty string if an exact match is not available.
  • When using a diff as a patch, and inserting new cells, a blank cell in the diff (either the NULL value or an empty string) should be treated as ambiguous, and the “right” thing done given the column type. If either value could be inserted, then the blank string should be inserted (since the encoded string NULL is available to specifically identify the NULL value).

Note that if the diff is being expressed in a table that allows nested structure (e.g. a JSON representation), a list representation for modified cells might be used to avoid this issue. There is no specification for that at this time.

# Expressing a moved row

TIP

This can be ignored for tables for which row order is meaningless, e.g. in typical relational databases.

A row that have been moved in a table for which row order is meaningful is marked with a : tag in the action column and placed in the order it appears in the REMOTE table.

To avoid burdening human readers with too much arcana, tags are not combined when multiple kinds of actions apply to a row or column. So for example, a context row that was moved and had a cell added will not be tagged as :+ or +: or such-like, but rather by :. Cell addition can be determined from the schema row. These weak tags are included as aids for highlighting to express the most significant thing to know about a row.

# Expressing a moved column

TIP

This can be ignored for tables for which column order is meaningless.

A column that have been moved in a table for which column order is meaningful is marked with a : tag in the schema row and placed in the order it appears in the REMOTE table.

If a diff that contains a : tag is used to patch a table for which column order is not meaningful, that tag should simply be ignored.

# Reference: action column tags

Symbol Meaning
@@ The header row, giving column names.
! The schema row, given column differences.
+++ An inserted row (present in REMOTE, not present in LOCAL).
--- A deleted row (present in LOCAL, not present in REMOTE).
-> A row with at least one cell modified cell. -->, --->, ----> etc. have the same meaning.
Blank A blank string or NULL marks a row common to LOCAL and REMOTE, given for context.
... Declares that rows common to LOCAL and REMOTE are being skipped.
+ A row with at least one added cell.
: A reordered row.

# Reference: schema row tags

Symbol Meaning
+++ An inserted column (present in REMOTE, not present in LOCAL).
--- A deleted column (present in LOCAL, not present in REMOTE).
(<NAME>) A renamed column (the name in LOCAL is given in parenthesis, and the name in REMOTE will be in the header row).
Blank A blank string or NULL marks a column common to LOCAL and REMOTE, given for context.
... Declares that columns common to LOCAL and REMOTE are being skipped.
: A reordered column.
- + diff --git a/views/index.html b/views/index.html index 2f0dbae5..e7fd4520 100644 --- a/views/index.html +++ b/views/index.html @@ -8,8 +8,8 @@ - - + +
Frictionless Standards
- +