Ko Mobiletto he paparanga tangohanga rokiroki JavaScript, me te whakamunatanga taha-kiritaki marama.
- [He aha a Mobiletto?](#He aha-Mobiletto?)
- Tīmatatanga tere
- Mobiletto CLI
- Putake
- Tāuta
- Tautoko me te Putea
- [Whakamahinga Taketake](#Whakamahinga Taketake)
- Metadata
- Kāhua kawemai kē
- Keteroki
- Whakaata
- [Whakamunatanga Puata](#Whakamunatanga Puata)
- Te hurihanga matua
- [Atanga atekōkiri](#Atanga Atekōkiri)
- Takitaki
Kua whakamaoritia tenei tuhinga README.md, ma te hokeylization, ki roto nga reo katoa e tautokohia ana e Google Translate!
E mohio ana ahau ehara i te mea tino tika, engari ko taku tumanako he pai ake i te kore!
🇸🇦 Arapi 🇧🇩 Bengali 🇩🇪 Tiamana 🇺🇸 Ingarihi 🇪🇸 Paniora 🇫🇷 Wīwī 🇹🇩 Hausa 🇮🇳 Hindi 🇮🇩 Indonesian 🇮🇹 Itari 🇯🇵 Japanese 🇰🇷 Korean 🇮🇳 Marathi 🇵🇱 Polish 🇧🇷 Potiti 🇷🇺 Ruhia 🇰🇪 Swahili 🇵🇭 Tagalog 🇹🇷 Turkish 🇵🇰 Urdu 🇻🇳 Vietnamese 🇨🇳 Hainamana
Ko tenei whakamaoritanga o te [README] taketake(https://github.com/cobbzilla/mobiletto/blob/master/README.md) kei te he -- e tino manakohia nga whakatikatika! Tukuna mai he tono toia ki a GitHub, ki te kore koe e pai ki te mahi i tera, whakatuwhera i tetahi take
Ina hanga e koe he take GitHub hou mo te whakamaoritanga, mahia:
- whakauruhia te URL wharangi (kape/whakapiri mai i te pae wahitau tirotiro)
- whakaurua te tuhinga tika e he ana (tārua/whakapiri mai i te kaitirotiro)
- whakaahuatia mai he aha te he -- kei te he te whakamaoritanga? kua pakaru te whakatakotoranga?
- tuku mai he whakaaro mo te whakamaoritanga pai ake, me pehea ranei te whakatakotoranga tika o te tuhinga
- Mauruuru!
Ko nga momo kaiwhakarato rokiroki kapua he API hotokore. Ahakoa te hunga e ngana ana ki te "hototahi S3" he whanonga motuhake.
Ina whiriwhiria e koe tetahi kaihoko rokiroki mo to taupānga, mena ka tohu tika koe ki to API, to taupānga kei te whakawhirinaki inaianei ki tera ratonga. Ka haere te wa, ka whakaemihia te waehere, ka huri nga kaihoko e kore e taea. Nau mai ki te ao ngahau o te raka kaihoko!
I hangaia a Mobiletto ki te whakaoti i tenei raru. Ma te whakawaehere i to taupānga ki te API a mobiletto, ka taea e koe Hurihia nga kaiwhakarato rokiroki me te mohio ka rite te ahua o te paparanga rokiroki o to taupānga.
Ka whakamatautauhia nga taraiwa katoa mo te whanonga rite me te 60+ whakamatautau mo ia taraiwa. Ka whakamatauhia e matou nga taraiwa katoa me nga huinga katoa o:
- Whakamunatanga: kua whakahohea me te mono
- Redis cache: kua whakahohea me te mono
Ma tenei huarahi ka mau te rongo ki a maatau ka rite tonu te mahi a mobiletto ahakoa ko wai te taraiwa e whakamahia ana e koe, ahakoa ka taea e koe te keteroki me te whakamunatanga/ranei.
Nga taraiwa rokiroki Mobiletto o naianei:
s3
: Amazon S3b2
: Backblaze B2local
: pūnaha kōnae rohe
Ko nga takoha hei tautoko i te maha atu o nga kaiwhakarato rokiroki kapua e tino manakohia ana!
Ko te Mobiletto ka whakamahia hei whare pukapuka e etahi atu waehere JavaScript.
Hei mahi me te mobiletto i te raina whakahau, whakamahia mobiletto-cli
Kei te ngana ahau ki te noho hei kaiwhakawhanake rorohiko puna tuwhera ngaio. I mahi ahau i roto te ahumahi rorohiko mo nga tau maha, kua timata ahau i nga kamupene angitu me te hoko atu ki nga kamupene a te iwi. No tata nei ka ngaro taku mahi, a karekau he mahi ke atu i te rarangi
Na ka ngana ahau ki te tuhi i nga raupaparorohiko awhina me te kite mena ka mahi tera
Mena he pai ki a koe te whakamahi i tenei rorohiko, ka tino mihi ahau mo te iti rawa koha-marama ma Patreon
Mauruuru!
Tāuta mā te whakamahi i te npm
, i te yarn
miroranei. Kei te pirangi pea koe ki te putanga
lite` e uru katoa nga
i whakamaoritia nga konae README:
npm install mobiletto-lite
yarn add mobiletto-lite
Mena kei te pirangi koe ki nga konae README i roto i nga reo katoa, whakauruhia te putanga katoa:
npm install mobiletto
yarn add mobiletto
He tauira poto ma te whakamahi i te taraiwa mobiletto s3
.
He rite tonu te mahi o tenei waehere mena he b2
, he b2
ranei te local
.
const storage = require('mobiletto')
const bucket = await storage.connect('s3', aws_key, aws_secret, {bucket: 'bk'})
// list objects: returns array of metadata objects
const listing = await bucket.list()
const dirList = await bucket.list('some/dir/')
const everything = await bucket.list('', {recursive: true})
// write an entire file
let bytesWritten = await bucket.writeFile('some/path', someBufferOfData)
// write a file from a stream/generator
bytesWritten = await bucket.write('some/path', streamOrGenerator)
// read an entire file
// returns null if an exception would otherwise be thrown
const bufferOrNull = await bucket.safeReadFile('some/path')
// stream-read a file, passing data to callback
const bytesRead = await bucket.read('some/path', (chunk) => { ...do something with chunk... } )
// remove a file, returns the path removed
let removed = await bucket.remove('some/path') // removed is a string
// remove a directory, returns array of paths removed
removed = await bucket.remove('some/directory', {recursive: true}) // removed is now an array!
He tauira tino whanui ake, e whakaatu ana i te nuinga o nga ahuatanga e tukuna ana:
const { mobiletto } = require('mobiletto')
// General usage
const api = await mobiletto(driverName, key, secret, opts)
// To use 'local' driver:
// * key: base directory
// * secret: ignored, can be null
// * opts object:
// * readOnly: optional, never change anything on the filesystem; default is false
// * fileMode: optional, permissions used when creating new files, default is 0600. can be string or integer
// * dirMode: optional, permissions used when creating new directories, default is 0700. can be string or integer
const local = await mobiletto('local', '/home/ubuntu/tmp', null, {fileMode: 0o0600, dirMode: '0700'})
// To use 's3' driver:
// * key: AWS Access Key ID
// * secret: AWS Secret Key
// * opts object:
// * readOnly: optional, never change anything on the bucket; default is false
// * bucket: required, name of the S3 bucket
// * region: optional, the AWS region to communicate with, default is us-east-1
// * prefix: optional, all read/writes within the S3 bucket will be under this prefix
// * delimiter: optional, directory delimiter, default is '/' (note: always '/' when encryption is enabled)
const s3 = await mobiletto('s3', aws_key, aws_secret, {bucket: 'bk', region: 'us-east-1'})
// To use 'b2' driver:
// * key: Backblaze Key ID
// * secret: Backblaze Application Key
// * opts object:
// * readOnly: optional, never change anything on the bucket; default is false
// * bucket: required, the ID (**not the name**) of the B2 bucket
// * prefix: optional, all read/writes within the B2 bucket will be under this prefix
// * delimiter: optional, directory delimiter, default is '/' (note: always '/' when encryption is enabled)
// * partSize: optional, large files will be split into chunks of this size when uploading
const b3 = await mobiletto('b2', b2_key_id, b2_app_key, {bucket: 'bk', partSize: 10000000})
// List files
api.list() // --> returns an array of metadata objects
// List files recursively
api.list({ recursive: true })
// List files in a directory
const path = 'some/path'
api.list(path)
api.list(path, { recursive: true }) // also supports recursive flag
// Visit files in a directory -- visitor function must be async
api.list(path, { visitor: myAsyncFunc })
api.list(path, { visitor: myAsyncFunc, recursive: true })
// The `list` method throws MobilettoNotFoundError if the path does not exist
// When you call `safeList` on a non-existent path, it returns an empty array
api.safeList('/path/that/does/not/exist') // returns []
// Read metadata for a file
api.metadata(path) // returns metadata object
// The `metadata` method throws MobilettoNotFoundError if the path does not exist
// When you call `safeMetadata` on a non-existent path, it returns null
api.safeMetadata('/tmp/does_not_exist') // returns null
// Read a file
// Provide a callback that writes the data someplace
const callback = (chunk) => { ... write chunk somewhere ... }
api.read(path, callback) // returns count of bytes read
// Read an entire file at once
const data = await api.readFile(path) // returns a byte Buffer of the file contents
// Read an entire file at once
// returns null if an exception would otherwise be thrown
const bufferOrNull = await bucket.safeReadFile('some/path')
// Write a file
// Provide a generator function that yields chunks of data
const generator = function* () {
while ( ... more-data-to-return ... ) {
data = ... load-data ...
yield data
}
}
local.api(path, generator) // returns count of bytes written
// Write an entire file at once (convenience method)
await api.writeFile(path, bufferOrString) // returns count of bytes written
// Delete a file
// Quiet param is optional (default false), when set errors will not be thrown if the path does not exist
// Always returns a value or throws an error.
// Return value may be a single string of the file removed, or an array of all files removed (driver-dependent)
const quiet = true
api.remove(path, {quiet}) // returns single path removed
// Recursively delete a directory and do it quietly (do not report errors)
const recursive = true
const quiet = true
api.remove(path, {recursive, quiet}) // returns array of paths removed
Ko te tono metadata
he whakahoki metadata mo te whakaurunga punaha konae kotahi.
Waihoki, ko te uara whakahoki mai i te tono list
he huinga ahanoa metadata.
He penei te ahua o te ahanoa metadata:
{
"name": "fully/qualified/path/to/file",
"type": "entry-type",
"size": size-in-bytes,
"ctime": creation-time-epoch-millis,
"mtime": modification-time-epoch-millis
}
Ko te āhuatanga type
ka taea te file
, dir
, link
, te special
ranei.
I runga i te momo atekōkiri, karekau pea te tono list
e whakahoki āpure katoa. Ko te name
' me type
āhuatanga
me noho tonu. Ma te tono metadata
ka whakahoki mai i nga taonga katoa e waatea ana.
Kawemai i te kōwae hōkai-katoa ka whakamahi i te mahi connect
:
const storage = require('mobiletto')
const opts = {bucket: 'bk', region: 'us-east-1'}
const s3 = await storage.connect('s3', aws_key, aws_secret, opts)
const objectData = await s3.readFile('some/path')
He pai ake te mahi a Mobiletto me te keteroki redis .
Ka ngana a Mobiletto ki te hono atu ki tetahi tauira redis i te 127.0.0.1:6379
Ka taea e koe te whakakore i tetahi o enei:
- Tautuhia te
MOBILETTO_REDIS_HOST
env var, mobiletto hono ki konei hei utu mo te localhost - Tautuhia te
MOBILETTO_REDIS_PORT
env var, ka whakamahia tenei tauranga
Ka rokirokia e Mobiletto ana taviri redis katoa me te prefix _mobiletto__
. Ka taea e koe te huri i tenei
mā te tautuhi i te MOBILETTO_REDIS_PREFIX
env var.
Ka taea hoki te whakarite keteroki hononga-hononga me te ahanoa opts.redisConfig
:
const redisConfig = {
enabled: true, // optional, default is true. if false other props are ignored
host: '127.0.0.1',
port: 6379,
prefix: '_mobiletto__'
}
const opts = { redisConfig, bucket: 'bk', region: 'us-east-1' }
const s3 = await storage.connect('s3', aws_key, aws_secret, opts)
Hei mono: tukuna enabled: false
ki to ahanoa opts.redisConfig
ina whakau koe i to hononga.
Ka rite ki te korero i raro nei, ko te whakakore i te keteroki ka pa he kino ki te mahi me te nui ake o nga tono ki te rokiroki e tino hiahia ana koe.
Pupuri whakamuna: te panui/tuhi he puhoi ake te putunga whakamuna i te tikanga, engari ko te whakatere huri noa i nga raarangi (he mahi etahi) he tino utu. Te whakamahi i te keteroki redis ka hoatu e koe he whakanui mahi nui.
He haumaru te keteroki taunoa, engari kare e pai ana mena he maha nga mahi tuhi/tango. Ko nga mahi tuhi, tango ranei ka whakakore i te keteroki katoa, me te whakarite ka kite nga panui ka whai ake huringa hou.
Mena kei te whakamahi koe i tetahi taputapu CLI penei i a mobiletto-cli,
Ka tino hiahia koe kia whakahohea te keteroki redis, na te mea ka mau tonu i nga tono o te whakahau mo
.
// Copy a local filesystem mobiletto to S3
s3.mirror(local)
// Mirror a local subdirectory from one mobiletto to an S3 mobiletto, with it's own subdirectory
local.mirror(s3, 'some/local-folder', 'some/s3-folder')
Ko te whakahau mirror
he mahi kape kotahi-wa o nga konae katoa mai i tetahi waea pūkoro ki tetahi atu.
Karekau he tukanga hei pupuri i te whakaata i roto i te waa. Whakahaerehia ano te tono mirror
ki te tukutahi i nga konae ngaro.
Ko te uara whakahoki mai i mirror
' he ahanoa ngawari me nga porotiti mo te maha o nga konae i angitu
i whakaatahia me te maha o nga konae i hapa:
{
success: count-of-files-mirrored,
errors: count-of-files-with-errors
}
WHAKATŪPATO: Ko te whakaata i nga huinga raraunga nui ka pau te waa me te whakapau kaha
Ki te mirror
karanga semantics i etahi wa ka raruraru te mohio ko wai te
te kaipānui me wai te kaituhi. Whakaarohia he korero taumahi: te "maui-maui mobiletto"
Ko te mea e tukuna ana ki te (whakaata i nga raraunga kua tuhia), me te "matau-matau mobiletto" (te
tohenga ki te tikanga mirror
) ko te uara e tautapa ana (ka panuitia nga raraunga whakaata).
Whakahohea te whakamunatanga taha-kiritaki marama:
// Pass encryption parameters
const encryption = {
// key is required, must be >= 16 chars
key: randomstring.generate(128),
// optional, the default is to derive IV from key
// when set, IV must be >= 16 chars
iv: randomstring.generate(128),
// optional, the default is aes-256-cbc
algo: 'aes-256-cbc'
}
const api = await mobiletto(driverName, key, secret, opts, encryption)
// Subsequent write operations will encrypt data (client side) when writing
// Subsequent read operations will decrypt data (client side) when reading
Kei te aha? He whaiaronga "whakaurunga whaiaronga" (rere) motuhake (whakamuna) e whai ana he aha nga konae kei roto whaiaronga (aka te whaiaronga whaiaro).
- Ko te tono
list
ka panui i nga konae whakaurunga whaiaronga, ka wetemuna ia ara kua whakarārangitia; ka whakahoki metadata mo ia kōnae - Ko nga tono
list
he koretake ake, ina koa mo nga whaiaronga he maha nga konae - Ko te tono
write
he tuhi i nga konae rereke ki roto i te raarangi whaiaronga o ia matua, he recursively; ka tuhia te kōnae - Ko nga tono
write
ka puta he O(N) tuhi, me te N = te hohonu i roto i te rarangi whaiaronga. - Ko te tono
remove
ka tango i te konae dirent e rite ana, me tona matua ki te putua, ka hoki whakamuri; ka tango i te konae - Ko nga tono
remove
-kore ka puta te O(N) panui me te maha o nga mukunga, me te N = te hohonu i roto i te raarangi whaiaronga. - He utu nui nga whakahau
remove
i runga i nga punaha konae nui me te hohonu
Kia mahara ahakoa kua whakahohea te whakamunatanga taha-kiritaki, he hoa riri e tino kitea ana ki to taha-tumau whakamuna. te rokiroki, ahakoa kaore he ki, ka taea tonu te kite i te tapeke o nga raarangi whaiaronga me te maha o nga konae kei roto i ia waahanga, me era atu kia kaha, kimihia etahi, te katoa ranei o te hanganga katoa o te rarangi whaiaronga. Tuhipoka: Whakamahia he hanganga ahua papatahi kia pai ake te haumarutanga. Kare te hoa riri e mohio ki nga ingoa o nga raarangi me nga konae mena ka mohio ano ratou ki to whakamunatanga kī, i momoho rānei te whakamunatanga. Ka mutu nga peti katoa!
Ko nga mahi i runga i te rokiroki whakamunatia ka puhoi. Ko nga rarangi recursive me nga tangohanga ka tino puhoi. He nui te awhina o te keteroki ma te redis, engari kia mohio kei te peehia te keteroki ki runga i nga tuhi ka tangohia ranei.
Waihangatia he waea pūkoro me to taviri hou, ka whakaatahia nga raraunga tawhito ki roto:
const storage = require('mobiletto')
const oldEncryption = { key: .... }
const oldStorage = await storage.connect('s3', aws_key, aws_secret, {bucket: 'bk', region: 'us-east-1'}, oldEncryption)
const newEncryption = { key: .... }
const newStorage = await storage.connect('s3', aws_key, aws_secret, {bucket: 'zz', region: 'us-east-1'}, newEncryption)
newStorage.mirror(oldStorage) // if oldStorage is very large, this may take a looooooong time...
Ko te taraiwa he konae JS ka kaweake i te mahi 'storageClient' me tenei waitohu:
function storageClient (key, secret, opts)
key
: he aho, ko to API matua (mo te atekōkirilocal
koinei te whaiaronga turanga)secret
: he aho, to API ngaro (ka taea te waiho mo te taraiwalocal
)opts
: he ahanoa, ko ngā āhuatanga mō ia atekōkiri:- Mo te
local
, ma ngafileMode
me tedirMode
e whakatau me pehea te hanga i nga konae hanga hou me nga whaiaronga. - Mo
s3
, me hiahia te taongabucket
. Ko nga ahuatanga whiriwhiri ko:region
: the S3 region, default is us-east-1prefix
: a prefix to prepend to all S3 paths, default is the empty stringdelimiter
: the directory delimiter, default is '/'
Ko te ahanoa ka whakahokia mai e te mahi storageClient me tautuhi enei mahi:
// Test the driver before using, ensure proper configuration
async testConfig ()
// List files in path (or from base-directory)
// If recursive is true, list recursively
// If visitor is defined, it will be an async function. await the visitor function on each file found
// Otherwise, perform the listing and return an array of objects
async list (path, recursive = false, visitor = null) // path may be omitted
// Read metadata for a path
async metadata (path)
// Read a file
// callback receives a chunk of data. endCallback is called at end-of-stream
async read (path, callback, endCallback = null)
// Write a file
// driver must be able to handle a generator or a stream
async write (path, generatorOrReadableStream)
// Remove a file, or recursively delete a directory
// returns a string of a single path removed, or an array of multiple paths removed
async remove (path, recursive = false, quiet = false)
Ka whakamahi a Mobiletto i te whare pukapuka takiuru winston.
Kei roto i nga rangitaki ** nga ara konae me nga karere hapa, engari ka **kaore e mau nga taviri, nga mea ngaro, etahi atu korero whirihoranga hononga ranei.
Whakamahia te taurangi taiao MOBILETTO_LOG_LEVEL
ki te tautuhi i te taumata rangitaki, ma te whakamahi i tetahi
o nga taumata npm
kua tautuhia ki https://www.npmjs.com/package/winston#logging-levels
Ko te taumata taunoa ko error
. Ko te taumata tino kupu ko te silly
, ahakoa he motini i tenei wa
karekau e takiuru ki nga taumata i raro iho i debug
MOBILETTO_LOG_LEVEL=silly # maximum logs!
Ma te taunoa, ka tuhi te kairakau ki te papatohu. Hei tuku rangitaki ki tetahi konae, tautuhia te MOBILETTO_LOG_FILE
taurangi taiao. I te wa e takiuru ana ki tetahi konae, ka kore e tuhia nga raarangi ki te papatohu.
MOBILETTO_LOG_FILE=/var/my_mobiletto_log
Hei whakaweto i te takiuru:
MOBILETTO_LOG_FILE=/dev/null