You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I have a SQLite table like so (simplified schema below) with ~950K rows
CREATE TABLE t (id INTEGER PRIMARY KEY, fulltext TEXT);
What is the quickest and easiest way to generate embeddings and store them in a vectors table? I am using Ollama with llama 3.2 running locally using the nomic-embed-text embeddings model. I would like to do this once and then create a web interface to query the data. Additionally, as new rows get added to the table t, I would like to TRIGGER embeddings for them as well. Is that possible? If yes, any hint would be very welcome.
The text was updated successfully, but these errors were encountered:
Is there a way to use transactions when inserting a lot of embeddings in a libSQL db? Right now I am using a pattern like so, and it is really slow (I have about a million documents for which I need to insert embeddings). I'd like to insert them in transactions of 5000 at a time
const rows = db.prepare(`SELECT fulltext FROM t LIMIT 5000`).all();
for (const row of result.rows) {
await app.addLoader(new TextLoader({ text: row.fulltext }));
}
I have a SQLite table like so (simplified schema below) with ~950K rows
What is the quickest and easiest way to generate embeddings and store them in a
vectors
table? I am using Ollama with llama 3.2 running locally using thenomic-embed-text
embeddings model. I would like to do this once and then create a web interface to query the data. Additionally, as new rows get added to the tablet
, I would like toTRIGGER
embeddings for them as well. Is that possible? If yes, any hint would be very welcome.The text was updated successfully, but these errors were encountered: