diff --git a/_posts/digests/2023-10-23-242.md b/_posts/digests/2023-10-23-242.md
index 5442ec87..3286998d 100644
--- a/_posts/digests/2023-10-23-242.md
+++ b/_posts/digests/2023-10-23-242.md
@@ -18,14 +18,18 @@ redirect: https://lastweekin.ai/p/242
Amazon is set to begin testing Agility's bipedal robot, Digit, in its nationwide fulfillment centers, marking a significant step in the application of humanoid robots in industrial settings. This follows Amazon's inclusion of Agility as one of the first five recipients of its $1 billion Industrial Innovation Fund. While Amazon Robotics has primarily focused on wheeled autonomous mobile robots (AMRs), the company is exploring the potential of legged locomotion, particularly for navigating diverse terrains. The integration of humanoid robots into Amazon's operations could significantly impact the trajectory of the robotics industry, particularly if they prove successful at scale. However, the company is also considering other mobile manipulation solutions, such as mounting a robot arm on an AMR. The success or failure of the Digit pilots could have far-reaching implications for the future of bipedal robots.
+#### [State of AI Report 2023](https://www.stateof.ai/)
+
+The State of AI Report 2023 highlights the dominance of Large Language Models (LLMs) in AI research, with significant advances in transformers surprising the AI community. The report discusses the rise of OpenAI's GPT-4 and the increasing reliance on computational power, alongside the thriving open-source community. However, the report also notes new tensions around openness due to commercial and safety concerns. Despite the focus on LLMs, the report also covers progress in other AI fields like navigation, weather prediction, self-driving cars, and music generation. Key takeaways include GPT-4's dominance, efforts to clone or surpass proprietary performance, real-world breakthroughs driven by LLMs and diffusion models, the importance of compute power, the rise of generative AI applications, the mainstreaming of the safety debate, and the challenges in evaluating state-of-the-art models.
#### [Adept Releases Fuyu-8B for Multimodal AI Agents](https://analyticsindiamag.com/adept-releases-fuyu-8b-for-multimodal-ai-agents/)
Adept has launched Fuyu-8B, a scaled-down version of their multimodal AI model, designed to understand charts, documents, and diagrams with improved OCR capabilities. The model, which is now accessible through HuggingFace, offers a simplified architecture and training process, making it more accessible and scalable. Fuyu-8B is tailored for digital AI agents, excelling in handling arbitrary image resolutions, answering queries related to graphs, diagrams, and UI-based questions, and delivering responses for large images in under 100 milliseconds. Despite its optimization for specific applications, it performs well in standard image understanding benchmarks. The model uses a vanilla decoder-only transformer, eliminating the need for a separate image encoder and simplifying its structure. In evaluations on prominent image-understanding datasets, Fuyu-8B demonstrated robust performance, outperforming models like QWEN-VL and PALM-e-12B on multiple metrics.
+#### [4K4D: Real-Time 4D View Synthesis at 4K Resolution](https://zju3dv.github.io/4k4d/)
-
+This paper proposes a new method for real-time view synthesis of dynamic 3D scenes at 4K resolution, called 4K4D. The method uses a 4D point cloud representation that supports hardware rasterization, resulting in faster rendering speeds. The authors also introduce a hybrid appearance model that enhances rendering quality while maintaining efficiency. They also develop a differentiable depth peeling algorithm to effectively learn the model from RGB videos. The method can render novel view videos at over 400 FPS on the DNA-Rendering dataset at 1080p resolution and 80 FPS on the ENeRF-Outdoor dataset at 4K resolution using an RTX 4090 GPU, which is 30x faster than previous methods, achieving state-of-the-art rendering quality.
### Other News
#### Applications
diff --git a/_posts/digests/2023-10-30-243.md b/_posts/digests/2023-10-30-243.md
new file mode 100644
index 00000000..c046216c
--- /dev/null
+++ b/_posts/digests/2023-10-30-243.md
@@ -0,0 +1,159 @@
+---
+layout: redirect
+title: "Last Week in AI #243"
+excerpt: "White House's sweeping AI order 📜, Cruise's self-driving license revoked 🚫🚗, Artists fight back against generative AI with Nightshade 🎨🤖, and more!"
+image:
+ feature: assets/img/digests/243/1698171046083-gettyimages-1556779089.jpeg?image-resize-opts=Y3JvcD0xeHc6MC44NDI1eGg7MHh3LDAuMDUxN3hoJnJlc2l6ZT0xMjAwOiomcmVzaXplPTEyMDA6Kg
+ credit: /
+categories: [digests]
+permalink: /digests/the-two-hundred-and-forty-third
+sidebartoc: true
+redirect: https://lastweekin.ai/p/243
+---
+
+### Top News
+
+#### [White House to unveil sweeping AI executive order next week, tackling immigration, safety](https://www.washingtonpost.com/technology/2023/10/25/artificial-intelligence-executive-order-biden/)
+
+
+The Biden administration is set to announce a comprehensive executive order on artificial intelligence (AI), marking the U.S. government's most significant attempt to regulate this rapidly evolving technology. The order will require advanced AI models to undergo assessments before federal workers can use them, and it will also ease immigration barriers for highly skilled workers to boost the U.S.'s technological edge. The order comes as the European Union and other governments are working to regulate the riskiest uses of AI. The executive order is expected to build on voluntary commitments signed by 15 companies, including OpenAI, Google, Adobe, and Nvidia, to develop technology to identify AI-generated images and share safety data with the government and academics. The assessments, known as "red teaming," will likely be led by the National Institute of Standards and Technology (NIST). The order also includes provisions to change the immigration process and requires federal agencies to assess the current size of the AI workforce.
+
+
+#### [Cruise Self-Driving License Revoked After It Withheld Pedestrian Injury Footage, DMV Says](https://www.vice.com/en/article/4a3ba3/california-dmv-suspends-cruises-self-driving-car-license-after-pedestrian-injury)
+
+
+Cruise, one of the two autonomous vehicle companies offering fully driverless taxi rides in the U.S., has had its license revoked by the California Department of Motor Vehicles (DMV). The revocation was due to Cruise withholding video footage from an investigation into an incident where a Cruise vehicle ran over a pedestrian and then attempted a "pullover maneuver" while the pedestrian was still underneath the vehicle. The DMV only learned of the full extent of the incident from another government agency. Despite Cruise's spokesperson claiming that they had shown the full video to the DMV, the suspension remains effective immediately. Cruise can still operate in driverless mode but with a safety driver. The company has paused operations on driverless cars in San Francisco as a result of the suspension.
+
+
+#### [This new data poisoning tool lets artists fight back against generative AI](https://www.technologyreview.com/2023/10/23/1082189/data-poisoning-artists-fight-generative-ai/)
+
+
+The article discusses a new data poisoning tool, Nightshade, designed to help artists protect their work from being used without consent by generative AI models. The tool, while potentially open to misuse, would require thousands of poisoned samples to significantly impact larger models. Experts, such as Vitaly Shmatikov from Cornell University and Gautam Kamath from the University of Waterloo, highlight the need for robust defenses against such attacks and the importance of addressing vulnerabilities in AI models. The tool is seen as a potential deterrent for AI companies, encouraging them to respect artists' rights and possibly prompting them to pay royalties. Artists, such as Eva Toorenent and Autumn Beverly, express hope that Nightshade will shift the power balance back to the creators, protecting their work from unauthorized use.
+
+
+
+### Other News
+#### Tools
+
+
+[Jina AI Launches World's First Open-Source 8K Text Embedding, Rivaling OpenAI](https://jina.ai/news/jina-ai-launches-worlds-first-open-source-8k-text-embedding-rivaling-openai) - Jina AI has launched jina-embeddings-v2, an open-source text embedding model with an 8K context length, rivaling OpenAI's text-embedding-ada-002 in terms of capabilities and performance.
+
+[Qualcomm’s next big Snapdragon chip has leaked, and it’s full of AI features](https://www.theverge.com/2023/10/23/23928452/snapdragon-8-gen-3-leak-specs-ai-features-qualcomm) - Qualcomm's upcoming Snapdragon 8 Gen 3 chip for Android phones will have a heavy focus on AI, with features such as AI camera tools and the ability to run various AI models, making it a potential competitor for Google's Tensor processors.
+
+[Microsoft opens early access to AI assistant for infosec, Security Copilot](https://www.theregister.com/2023/10/23/microsoft_security_copilots_early_access/) - Microsoft is opening up the early access program for its flagship cybersecurity AI product, Security Copilot, which aims to save time and upskill security teams by providing step-by-step instructions on managing incidents and generating natural language reports.
+
+[Google Maps is becoming more like Search — thanks to AI](https://www.theverge.com/2023/10/26/23932315/google-maps-ai-immersive-view-ev-charging-search) - Google is adding new AI-powered features to Maps, including immersive navigation, easier-to-follow driving directions, and better organized search results, in an effort to make Maps more like Search and maintain a competitive edge over rivals like Apple and Microsoft.
+
+[Twelve Labs is building models that can understand videos at a deep level](https://techcrunch.com/2023/10/24/twelve-labs-is-building-models-that-can-understand-videos-at-a-deep-level/) - Twelve Labs is building AI models that can understand videos at a deep level, allowing developers to create apps that can search through videos, classify scenes, extract topics, and more, with potential applications in ad insertion, content moderation, media analytics, and automatic highlight reel generation.
+
+[Amazon now lets advertisers use generative AI to pretty up their product shots](https://www.theverge.com/2023/10/25/23931521/amazon-generative-ai-product-images-advertisers) - Amazon is beta testing AI image generation tools for advertisers, allowing them to easily create lifestyle imagery for their product ads, potentially leading to higher click-through rates.
+
+[Canva launches free AI 'Classroom Magic' tools for educators](https://mashable.com/article/canva-launches-classroom-ai-tools) - Canva has launched a suite of AI tools called Classroom Magic, designed to assist educators with lesson planning, content editing, document reformatting, image and text editing, multilingual lesson support, and accessibility checking.
+
+[Sick of meetings? Microsoft’s new AI assistant will go in your place](https://fortune.com/2023/10/19/microsoft-copilot-ai-assistant-teams-meetings/) - Microsoft has unveiled Copilot, an AI assistant that can attend meetings on behalf of employees, generating transcripts, summaries, and notes once the meeting is over, but some managers and workers are skeptical about the use of AI in meetings, citing the lack of nuanced human judgment and social skills needed for effective participation.
+
+#### Business
+
+
+[Lenovo and NVIDIA Announce Hybrid AI Solutions to Help Enterprises Quickly Adopt GenAI](https://www.businesswire.com/news/home/20231024169588/en/Lenovo-and-NVIDIA-Announce-Hybrid-AI-Solutions-to-Help-Enterprises-Quickly-Adopt-GenAI) - Lenovo and NVIDIA have announced an expansion of their partnership to bring generative AI to every enterprise, offering fully integrated systems that enable businesses to deploy tailored AI applications across various industries.
+
+[Wall Street is demanding financial results that support all the AI hype. Microsoft’s latest earnings finally delivered](https://fortune.com/2023/10/24/microsoft-earnings-q1-2024-ai-cloud-financial-results-wall-street/) - Wall Street is eager to see financial results that support the hype around AI, and Microsoft's latest earnings report, which showcased the business implications of AI adoption, delivered impressive numbers that exceeded expectations.
+
+[AI-based data center optimization startup MangoBoost raises $55M Series A](https://techcrunch.com/2023/10/23/ai-based-data-center-optimization-startup-mangoboost-raises-55m-series-a/) - AI-based data center optimization startup MangoBoost has raised $55 million in a Series A funding round to develop its DPU hardware and software solutions that help enterprises and data centers manage massive amounts of data to optimize workloads, with the goal of reducing power consumption and improving performance with cost efficiency and security.
+
+[AI Chip Startup Rebellions Is in Talks to Raise $100 Million](https://www.bloomberg.com/news/articles/2023-10-24/ai-chip-startup-rebellions-is-in-talks-to-raise-100-million) - AI chip startup Rebellions is in discussions to secure $100 million in funding to accelerate the advancement of their next-generation AI chip.
+
+[Intel’s in trouble as Nvidia and AMD reportedly prepare Arm-based desktop CPUs](https://fortune.com/2023/10/24/intel-nvidia-amd-desktop-arm-cpus-microsoft-qualcomm/) - Nvidia and AMD are reportedly working on Arm-based desktop CPUs, potentially posing a threat to Intel's dominance in the market.
+
+[GOOGLE, MICROSOFT, OPENAI, ANTHROPIC DRIVE FRONTIER MODEL FORUM'S DIRECTOR APPOINTMENT AND $10M AI FUND](https://yourstory.com/2023/10/ai-safety-frontier-model-forum-meserole-10m-fund) - The Frontier Model Forum has appointed Chris Meserole as its Executive Director and launched a $10 million AI Safety Fund, backed by tech giants like Google and Microsoft, to support independent researchers in innovating evaluation methods and "red teaming" tactics to ensure the safe development of advanced AI systems.
+
+[AI companies drive demand for office space in tech hubs, new study finds](https://finance.yahoo.com/news/ai-companies-drive-demand-for-office-space-in-tech-hubs-new-study-finds-142451573.html) - The boom in artificial intelligence is driving demand for office space in tech hubs, with AI companies seizing more office space, especially in the San Francisco Bay Area, as they look to grow quickly.
+
+[Meet The New AI Unicorns Of 2023](https://news.crunchbase.com/ai/new-ai-unicorns-2023-list-llm-platforms-cohere/) - One in five of the new billion-dollar startups to join The Crunchbase Unicorn Board in 2023 were AI companies, collectively adding $21 billion in value and dominated by generative AI companies in various sectors.
+
+[Luzia lands $10 million in funding to expand its WhatsApp-based chatbot](https://techcrunch.com/2023/10/20/luzia-lands-10-million-in-funding-to-expand-its-whatsapp-based-chatbot/) - Spain-based startup Luzia has raised $10 million in funding to expand its WhatsApp-based chatbot, which aims to introduce AI chatbot technology to non-tech affluent audiences in the Spanish and Portuguese-speaking market.
+
+[Stability AI General Counsel, HR Chief Depart From Startup](https://www.bloomberg.com/news/articles/2023-10-26/stability-ai-general-counsel-hr-chief-depart-from-startup) - Executive departures at an AI startup suggest further instability within the company.
+
+#### Research
+
+
+[Large language models propagate race-based medicine](https://www.nature.com/articles/s41746-023-00939-z) - Large language models (LLMs) integrated into healthcare systems may perpetuate harmful, race-based medicine by providing inaccurate and biased responses to medical questions related to race, according to a study assessing four commercially available LLMs.
+
+[AI ‘breakthrough’: neural net has human-like ability to generalize language](https://www.nature.com/articles/d41586-023-03272-3) - Scientists have created a neural network with human-like ability to make generalizations about language, which could lead to machines that interact with people more naturally and address the gaps and inconsistencies in current AI systems.
+
+[Researchers develop ‘Woodpecker’: A groundbreaking solution to AI’s hallucination problem](https://venturebeat.com/ai/researchers-develop-woodpecker-a-groundbreaking-solution-to-ais-hallucination-problem/) - Researchers have developed a framework called "Woodpecker" that corrects hallucinations in multimodal large language models (MLLMs) without the need for retraining, offering a promising solution to a significant problem in AI.
+
+[Proximal Policy Optimization (PPO): The Key to LLM Alignment](https://cameronrwolfe.substack.com/p/proximal-policy-optimization-ppo) - Recent AI research has shown that reinforcement learning (RL), specifically reinforcement learning from human feedback (RLHF), is crucial for training large language models (LLMs), and Proximal Policy Optimization (PPO) is a popular and effective RL algorithm used in the alignment of LLMs.
+
+[Contrastive Prefence Learning: Learning from Human Feedback without RL](https://arxiv.org/abs/2310.13639v1) - The article discusses Contrastive Preference Learning, a method of learning from human feedback without the use of Reinforcement Learning.
+
+[Branch-Solve-Merge Improves Large Language Model Evaluation and Generation](https://arxiv.org/abs/2310.15123?utm_source=aitidbits.substack.com&utm_medium=newsletter) - Branch-Solve-Merge (BSM) is a Large Language Model program that improves the evaluation and generation of text by decomposing tasks into sub-tasks and fusing their solutions, resulting in enhanced correctness, consistency, and coherence.
+
+[Exploring the Boundaries of GPT-4 in Radiology](https://arxiv.org/abs/2310.14573v1) - The article discusses the exploration of the boundaries of GPT-4 in the field of radiology.
+
+[SAM-CLIP: Merging Vision Foundation Models towards Semantic and Spatial Understanding](https://arxiv.org/abs/2310.15308v1) - The article discusses the merging of vision foundation models for semantic and spatial understanding in AI.
+
+[Towards Understanding Sycophancy in Language Models](https://arxiv.org/abs/2310.13548?utm_source=aitidbits.substack.com&utm_medium=newsletter) - The article discusses the issue of sycophancy in language models and presents recommendations for addressing it.
+
+[MusicAgent: An AI Agent for Music Understanding and Generation with Large Language Models](https://arxiv.org/abs/2310.11954v2) - MusicAgent is an AI agent that utilizes large language models to understand and generate music.
+
+[SOTOPIA: Interactive Evaluation for Social Intelligence in Language Agents](https://arxiv.org/abs/2310.11667v1) - SOTOPIA is an open-ended environment that simulates social interactions between artificial agents and evaluates their social intelligence, revealing significant differences between models and identifying challenging scenarios for AI systems.
+
+[Improving Wikipedia verifiability with AI](https://www.nature.com/articles/s42256-023-00726-1) - AI is being used to improve the verifiability of information on Wikipedia by developing tools that can retrieve and verify citations more accurately.
+
+[DreamSpace: Dreaming Your Room Space with Text-Driven Panoramic Texture Propagation](http://arxiv.org/abs/2310.13119) - The article discusses the importance of community support and donations for arXiv, an open access platform for scientific research.
+
+[FreeNoise: Tuning-Free Longer Video Diffusion Via Noise Rescheduling](https://arxiv.org/abs/2310.15169v1) - A study proposes FreeNoise, a tuning-free and time-efficient paradigm to enhance the generative capabilities of pretrained video diffusion models, allowing for the generation of longer videos conditioned on multiple text prompts.
+
+#### Concerns
+
+
+[AI risk must be treated as seriously as climate crisis, says Google DeepMind chief](https://www.theguardian.com/technology/2023/oct/24/ai-risk-climate-crisis-google-deepmind-chief-demis-hassabis-regulation) - The risks of artificial intelligence must be treated as seriously as the climate crisis, according to Demis Hassabis, the CEO of Google's AI unit, who called for the creation of an oversight body similar to the Intergovernmental Panel on Climate Change (IPCC) to address the dangers posed by AI, including the creation of bioweapons and the existential threat of super-intelligent systems.
+
+[Health providers say AI chatbots could improve care. But research says some are perpetuating racism](https://apnews.com/article/ai-chatbots-racist-medicine-chatgpt-bard-6f2a330086acd0a1f8955ac995bdde4d) - Popular AI chatbots used in healthcare are perpetuating racist and debunked medical ideas, potentially worsening health disparities for Black patients, according to a study by Stanford School of Medicine researchers.
+
+[AI-created child sexual abuse images ‘threaten to overwhelm internet’](https://www.theguardian.com/technology/2023/oct/25/ai-created-child-sexual-abuse-images-threaten-overwhelm-internet) - The Internet Watch Foundation has warned that artificial intelligence-generated child sexual abuse images are becoming a reality and pose a significant threat to the internet, with nearly 3,000 AI-made abuse images that broke UK law already identified.
+
+[Top AI Shops Fail Transparency Test](https://spectrum.ieee.org/ai-ethics) - Fifteen major AI companies have failed a transparency test despite signing on to the White House's commitments to manage AI risks.
+
+[AI researchers uncover ethical, legal risks to using popular data sets](https://www.washingtonpost.com/technology/2023/10/25/data-provenance/) - AI researchers have found that many popular data sets used to train generative AI systems are improperly licensed, with about 70% not specifying the correct license or being mislabeled, leading to confusion and potential copyright issues for developers.
+
+[Google Pixel’s face-altering photo tool sparks AI manipulation debate](https://www.bbc.com/news/technology-67170014) - Google Pixel's new face-altering photo tool, Best Take, uses machine learning to mix and match expressions from past photos, sparking a debate about AI manipulation.
+
+[Open-source AI firm Hugging Face confirms ‘regrettable accessibility issues’ in China](https://www.semafor.com/article/10/20/2023/ai-platform-hugging-face-confirms-china-blocked-it) - Hugging Face, an open-source AI firm, acknowledges accessibility issues in China after the country blocked access to its platform, but it remains unclear why the censorship occurred.
+
+[Misinformation reloaded? Fears about the impact of generative AI on misinformation are overblown](https://misinforeview.hks.harvard.edu/article/misinformation-reloaded-fears-about-the-impact-of-generative-ai-on-misinformation-are-overblown/) - Concerns about the impact of generative AI on the misinformation landscape are overblown, as evidence suggests that the increased quantity, quality, and personalization of misinformation are speculative and have limited effects on the spread of misinformation.
+
+[Is my co-worker AI? Bizarre product reviews leave Gannett staff wondering](https://www.theverge.com/2023/10/26/23931530/gannett-ai-product-reviews-site-reviewed-union-newsguild) - Gannett staff are questioning whether some product reviews on their site were written by AI or humans, with the company claiming they were created by third-party freelancers hired by a marketing agency partner.
+
+[AI Has a Hotness Problem](https://www.theatlantic.com/technology/archive/2023/10/ai-image-generation-hot-people/675750/) - AI image-generation tools tend to produce attractive faces because they are trained on databases of existing photos that are biased towards attractive people, and the tools themselves tend to generate faces that look like averaged faces, resulting in a phenomenon where AI-generated faces are often more attractive than real people.
+
+#### Analysis
+
+
+[How the Foundation Model Transparency Index Distorts Transparency](https://www.interconnects.ai/p/fmti-critique) - The Foundation Model Transparency Index (FMTI) released by Stanford's Center for Research on Foundation Models (CRFM) is criticized for misleadingly measuring transparency for foundation models and instead measuring how well-documented commercial products are, as well as making critical factual errors and being biased against openly released models.
+
+[Ideologies of Awe & AI Art at the MoMA](https://cyberneticforests.substack.com/p/ideologies-of-awe-and-ai-art-at-the) - Refik Anadol's "Unsupervised - Machine Hallucinations" at the MoMA uses AI to create visualizations of the museum's art archive, but the work unintentionally reinforces the idea that AI is complex and unknowable, leaving viewers passive and in awe of its power.
+
+#### Policy
+
+
+[AI Godfathers Bengio and Hinton: Major tech companies should devote a third of AI budget to managing AI risk](https://venturebeat.com/ai/ai-godfathers-bengio-and-hinton-major-tech-companies-should-devote-a-third-of-ai-budget-to-managing-ai-risk/) - Yoshua Bengio and Geoffrey Hinton, along with other AI experts, propose that major tech companies and governments allocate a third of their AI research and development budgets to AI safety and ethical use, in order to address the growing risks associated with artificial intelligence.
+
+[What the U.N.’s AI Advisory Group Will Do](https://time.com/6328861/un-advisory-group-ai/) - The U.N. has unveiled a new advisory body dedicated to developing consensus around the risks posed by artificial intelligence and how international cooperation can help meet those challenges, with the body's recommendations potentially deciding the form and function of a U.N. agency for the governance of AI.
+
+[UK to set up world's first AI safety institute, Sunak says](https://www.reuters.com/world/uk/uk-set-up-worlds-first-ai-safety-institute-sunak-says-2023-10-26/) - UK Prime Minister Rishi Sunak announces plans to establish the world's first AI safety institute, which will focus on advancing knowledge of AI safety and evaluating the risks associated with new AI models.
+
+[OpenAI forms new team to assess ‘catastrophic risks’ of AI](https://www.theverge.com/2023/10/26/23933783/openai-preparedness-team-catastrophic-risks-ai) - OpenAI is forming a new team to assess and mitigate the potential catastrophic risks associated with AI, including nuclear threats, chemical and biological threats, autonomous replication, AI tricking humans, and cybersecurity threats.
+
+[We don’t want to set up global AI regulator, says UK tech secretary](https://www.politico.eu/article/michelle-donelan-global-ai-regulator/) - The UK government does not plan to establish a global regulator for artificial intelligence, but instead aims to set up international networks and frameworks to manage risks through its AI safety summit and Frontier AI Taskforce.
+
+#### Fun
+
+
+[The Beatles: ‘final’ song Now and Then to be released thanks to AI technology](https://www.theguardian.com/music/2023/oct/26/the-beatles-final-song-now-and-then-ai-technology) - The long-awaited "final" Beatles song, Now and Then, featuring all four members, is set to be released next week thanks to AI technology that was used to enhance the audio on Peter Jackson's documentary Get Back.
+
+
+
+Copyright © 2023 Skynet Today, All rights reserved.
diff --git a/assets/img/digests/243/1698171046083-gettyimages-1556779089.jpeg?image-resize-opts=Y3JvcD0xeHc6MC44NDI1eGg7MHh3LDAuMDUxN3hoJnJlc2l6ZT0xMjAwOiomcmVzaXplPTEyMDA6Kg b/assets/img/digests/243/1698171046083-gettyimages-1556779089.jpeg?image-resize-opts=Y3JvcD0xeHc6MC44NDI1eGg7MHh3LDAuMDUxN3hoJnJlc2l6ZT0xMjAwOiomcmVzaXplPTEyMDA6Kg
new file mode 100644
index 00000000..417aecba
Binary files /dev/null and b/assets/img/digests/243/1698171046083-gettyimages-1556779089.jpeg?image-resize-opts=Y3JvcD0xeHc6MC44NDI1eGg7MHh3LDAuMDUxN3hoJnJlc2l6ZT0xMjAwOiomcmVzaXplPTEyMDA6Kg differ
diff --git a/assets/img/digests/243/GettyImages-1236495668-scaled.jpg b/assets/img/digests/243/GettyImages-1236495668-scaled.jpg
new file mode 100644
index 00000000..28f1dacc
Binary files /dev/null and b/assets/img/digests/243/GettyImages-1236495668-scaled.jpg differ
diff --git a/scripts/csv2md.py b/scripts/csv2md.py
index a0c6e9ec..c1fa3b63 100644
--- a/scripts/csv2md.py
+++ b/scripts/csv2md.py
@@ -1,6 +1,7 @@
import os
import argparse
import requests
+import multiprocessing
import pandas as pd
import inflect
@@ -18,10 +19,10 @@
CATEGORIES = [
'Top News',
- 'Applications',
+ 'Tools',
'Business',
- 'Concerns',
'Research',
+ 'Concerns',
'Analysis',
'Policy',
'Expert Opinions',
@@ -30,10 +31,26 @@
]
+def apply_map_batch(func, args_list):
+ pool = multiprocessing.Pool(os.cpu_count())
+ promises = [pool.apply_async(func, args) for args in args_list]
+
+ results = [None] * len(promises)
+ done_idxs = set()
+ pbar = tqdm(total=len(promises))
+ while len(done_idxs) < len(promises):
+ for idx, promise in enumerate(promises):
+ if idx not in done_idxs and promise.ready():
+ done_idxs.add(idx)
+ results[idx] = promise.get()
+ pbar.update(1)
+ return results
+
+
@retry(wait=wait_random_exponential(min=1, max=10), stop=stop_after_attempt(10))
def query_openai(messages, max_tokens=10, model='gpt-3.5-turbo-16k'):
return openai.ChatCompletion.create(
- model=model,
+ model=model,
messages=messages,
max_tokens=max_tokens,
temperature=0
@@ -54,9 +71,9 @@ def get_article_category(row, excerpt):
system_prompt = '''
Your task is to classify articles about AI into one of the following types:
-Business: Anything related to product announcements, investments, funding, VCs, company updates, or market trends.
+Business: Anything related to investments, funding, VCs, company updates, or market trends.
Research: Scientific studies, research in AI, or applying AI to do science in various fields. All links from arxiv and huggingface belong to Research.
-Applications: Applying AI to do something.
+Tools: New feature releases, product announcements; new AI software, tools, and applications of AI.
Concerns: Discussions and news about problems, harms, and any alarming things about AI, including govermnet investigations about AI.
Policy: News, analysis, and opinions related to government policies.
Analysis: Analyzes an existing topic about AI that's not the above topics (not news).
@@ -66,7 +83,7 @@ def get_article_category(row, excerpt):
The user will provide the article title, link, and description.
After careful consideration, you will respond with ONLY the predicted article type, with no explanations, punctuation, formatting, or anything else.
-Please only respond with one of the above types (Business, Resesarch, Applications, Concerns, Policy, Analysis, Expert Opinions, Explainers, Fun).
+Please only respond with one of the above types (Business, Resesarch, Tools, Concerns, Policy, Analysis, Expert Opinions, Explainers, Fun).
'''.strip()
return query_openai([
{'role': 'system', 'content': system_prompt},
@@ -83,7 +100,11 @@ def get_news_article(url):
article.download()
article.parse()
assert article.text
- return article
+ return {
+ 'text': article.text,
+ 'top_image': article.top_image,
+ 'has_top_image': article.has_top_image()
+ }
except:
return None
@@ -106,7 +127,7 @@ def get_article_excerpt(row, article):
prompt = f'''
Title: {row['Name']}
Subtitle: {row['Excerpt']}
-Text: {clip_text_words(article.text)}
+Text: {clip_text_words(article["text"])}
'''.strip()
return query_openai([
{'role': 'system', 'content': system_prompt},
@@ -170,7 +191,7 @@ def get_article_summary(title, news_article):
user_prompt = f'''
Title: {title}
-{clip_text_words(news_article.text)}
+{clip_text_words(news_article["text"])}
'''.strip()
messages = [
@@ -272,35 +293,43 @@ def get_newsletter_excerpt(top_news):
print(f'Reading {input_csv}')
csv = pd.read_csv(input_csv, encoding='utf-8')
- rows_news_articles = []
- for row_num, row in tqdm(csv.iterrows(), total=len(csv)):
+ rows = []
+ for row_num, row in csv.iterrows():
if 'arxiv' in row['URL']:
# remove "Title:" from arxiv titles
row['Name'] = row['Name'][6:]
if 'youtube' in row['URL']:
continue
+ rows.append(row)
- news_article = get_news_article(row['URL'])
- if not news_article:
- continue
-
- rows_news_articles.append((row, news_article))
+ print('Getting news articles...')
+ news_articles = apply_map_batch(
+ get_news_article, [(row['URL'],) for row in rows]
+ )
+ rows = [row for row, news_article in zip(rows, news_articles) if news_article]
+ news_articles = [news_article for news_article in news_articles if news_article]
print('Getting article excerpts...')
- excerpts = [
- get_article_excerpt(row, news_article)
- for row, news_article in tqdm(rows_news_articles)
- ]
+ excerpts = apply_map_batch(
+ get_article_excerpt,
+ [
+ (row, news_article)
+ for row, news_article in zip(rows, news_articles)
+ ]
+ )
print('Getting article categories...')
- categories = [
- get_article_category(row, excerpt)
- for (row, _), excerpt in tqdm(zip(rows_news_articles, excerpts), total=len(rows_news_articles))
- ]
+ categories = apply_map_batch(
+ get_article_category,
+ [
+ (row, excerpt)
+ for row, excerpt in zip(rows, excerpts)
+ ]
+ )
articles_map = {c : [] for c in CATEGORIES}
- for (row, news_article), excerpt, category in zip(rows_news_articles, excerpts, categories):
+ for row, news_article, excerpt, category in zip(rows, news_articles, excerpts, categories):
articles_map[category].append({
'url': row['URL'],
'title': row['Name'],
@@ -319,7 +348,7 @@ def get_newsletter_excerpt(top_news):
# place the first article w/ an image first
rank = rank_articles(articles)
for idx, r in enumerate(rank):
- if articles[r]['news_article'].has_top_image():
+ if articles[r]['news_article']['has_top_image']:
rank[0], rank[idx] = rank[idx], rank[0]
break
@@ -327,21 +356,29 @@ def get_newsletter_excerpt(top_news):
top_news += f'### {c}'
top_news += '\n\n'
+ summaries = apply_map_batch(
+ get_article_summary,
+ [
+ (article['title'], article['news_article'])
+ for article in articles
+ ]
+ )
+
for r in tqdm(rank, leave=False):
article = articles[r]
+ summary = summaries[r]
title, url, news_article = article['title'], article['url'], article['news_article']
- summary = get_article_summary(title, news_article)
top_news += f'#### [{title}]({url})'
top_news += '\n'
- if news_article.has_top_image():
- top_news += f''
+ if news_article['has_top_image']:
+ top_news += f'![]({news_article["top_image"]})'
if r == 0:
- im_response = requests.get(news_article.top_image)
+ im_response = requests.get(news_article['top_image'])
if im_response.status_code == 200:
- im_name = news_article.top_image.split("/")[-1]
+ im_name = news_article['top_image'].split("/")[-1]
with open(im_folder / im_name, "wb") as f:
f.write(im_response.content)
@@ -352,9 +389,9 @@ def get_newsletter_excerpt(top_news):
else:
content += f'#### {c}'
- if articles[rank[0]]['news_article'].has_top_image():
+ if articles[rank[0]]['news_article']['has_top_image']:
content += '\n'
- content += f'![]({articles[rank[0]]["news_article"].top_image})'
+ content += f'![]({articles[rank[0]]["news_article"]["top_image"]})'
content += '\n\n'
for r in tqdm(rank, leave=False):