diff --git a/_posts/digests/2024-08-26-285.md b/_posts/digests/2024-08-26-285.md new file mode 100644 index 00000000..d2be6566 --- /dev/null +++ b/_posts/digests/2024-08-26-285.md @@ -0,0 +1,177 @@ +--- +layout: redirect +title: "Last Week in AI #285" +excerpt: "Microsoft's Phi-3.5 outshines competitors, Nvidia's Llama-3.1-Minitron 4B packs a punch, AI21's Jamba models revolutionize enterprise AI, OpenAI's regulatory stance under fire, Dracarys models set coding ablaze, and more! 🚀🔥🤖" +image: + feature: assets/img/digests/285/03b85I3BA23x1ZUWUHJwU3F-1.fit_lim.size_1200x630.v1724412145.jpg + credit: / +categories: [digests] +permalink: /digests/the-two-hundred-and-eighty-fifth +sidebartoc: true +redirect: https://lastweekin.ai/p/285 +--- + +### Top News + +#### [Microsoft reveals Phi-3.5 — this new small AI model outperforms Gemini and GPT-4o](https://www.tomsguide.com/ai/microsoft-reveals-phi-35-this-new-small-ai-model-outperforms-gemini-and-gpt-4o) +![](https://cdn.mos.cms.futurecdn.net/ovzYvtXK2TceenUTxorprA-1200-80.jpg) + +Microsoft has unveiled Phi-3.5, the latest version of its small language model, which outperforms other small models from Google, OpenAI, Mistral, and Meta on several key metrics. Phi-3.5 is available in 3.8 billion, 4.15 billion, and 41.9 billion parameter versions, all of which can be downloaded for free and run using a local tool like Ollama. The model excels in reasoning and math benchmarks, surpassing competitors like Llama and Gemini. Phi-3.5 also comes in a vision model version that can understand images, and a mixture of expert models that split learning tasks across different sub-networks for more efficient processing. Despite some limitations in phrasing and simple tests, the model's architecture allows it to maintain efficiency while managing complex AI tasks in different languages, particularly in STEM and social sciences areas. + +More on this: + +#### [Nvidia’s Llama-3.1-Minitron 4B is a small language model that punches above its weight](https://venturebeat.com/ai/nvidias-llama-3-1-minitron-4b-is-a-small-language-model-that-punches-above-its-weight/) +![](https://venturebeat.com/wp-content/uploads/2024/04/image1_e12e57.png?w=1024?w=1200&strip=all) + +Nvidia's research team has developed a small language model (SLM), Llama-3.1-Minitron 4B, that performs comparably to larger models while being more efficient to train and deploy. The team used techniques of pruning and distillation to create this model. Pruning involves removing less important components of a model, while distillation transfers knowledge from a larger model to a smaller one. The team started with the Llama 3.1 8B model, fine-tuned it on a 94-billion-token dataset, applied depth-only and width-only pruning, and then fine-tuned the pruned models using NeMo-Aligner. The resulting Llama-3.1-Minitron 4B model performed close to other SLMs, despite being trained on a fraction of the training data. The width-pruned version of the model has been released on Hugging Face under the Nvidia Open Model License for commercial use. + + +More on this: + +#### [AI21 Introduces the Jamba Model Family: The most powerful and efficient long-context models for the enterprise](https://finance.yahoo.com/news/ai21-introduces-jamba-model-family-130000056.html) +![](https://media.zenfs.com/en/prnewswire.com/2738a6ded11043b7ccc2b037bd42ae49) + +AI21, a leading AI systems and foundation models developer, has announced the release of two new open models, Jamba 1.5 Mini and Jamba 1.5 Large. These models, built on a hybrid architecture that combines the strengths of Transformer and Mamba architectures, offer unmatched speed, efficiency, and performance in long-context language models. The Jamba 1.5 Large, a Mixture-of-Experts (MoE) model with 398B total parameters and 94B active parameters, is designed to handle complex reasoning tasks with high quality and efficiency. Both models utilize a true context window of 256K tokens, the largest currently available under an open license, and have demonstrated superior performance in latency tests against similar models. AI21 has partnered with major tech companies like Amazon Web Services, Google Cloud, Microsoft Azure, Snowflake, Databricks, and NVIDIA to ensure seamless deployment and utilization of the Jamba models in enterprise environments. + + +More on this: + +#### [Ex-OpenAI researchers claim Sam Altman's public support for AI regulation is a facade: "When actual regulation is on the table, he opposes it"](https://www.windowscentral.com/software-apps/ex-openai-researchers-claim-sam-altmans-public-support-for-ai-regulation-is-a-facade-when-actual-regulation-is-on-the-table-he-opposes-it) +![](https://cdn.mos.cms.futurecdn.net/Li6ishbDQkw9Po3Jkwcz5Z-1200-80.jpg) + +OpenAI, the creator of ChatGPT, has opposed a proposed AI bill (SB 1047) aimed at implementing safety measures to prevent AI technology from causing potential harm. This opposition has sparked criticism from former OpenAI researchers William Saunders and Daniel Kokotajlo, who argue that the development of advanced AI models without proper regulation could lead to catastrophic consequences. OpenAI CEO Sam Altman has publicly advocated for AI regulation, but the researchers suggest that his support is superficial, as he opposes actual regulatory measures when they are proposed. OpenAI's Chief Strategy Officer, Jason Kwon, has argued that AI regulation should be implemented at the federal level to foster innovation and establish global standards, but it remains uncertain whether the bill will be passed or if OpenAI's proposed amendments will be incorporated. + + +More on this: + +#### [Open source Dracarys models ignite generative AI fired coding](https://venturebeat.com/ai/open-source-dracarys-models-ignite-generative-ai-fired-coding/) +![](https://venturebeat.com/wp-content/uploads/2024/08/Dracarys-is-AI.png?w=1024?w=1200&strip=all) + +Abacus.ai, an AI model development platform, has introduced a new family of open large language models (LLMs) for coding, named Dracarys. Unlike its previous general-purpose LLM, Smaug-72B, Dracarys is specifically designed to optimize coding tasks. The "Dracarys recipe" has been applied to the 70B parameter class of models, involving optimized fine-tuning techniques to improve the coding abilities of any open-source LLM. According to LiveBench benchmarks, the Dracarys tuned version significantly improves the performance of existing models. Abacus.ai plans to release Dracarys versions for more models in the future, and the fine-tuned models are now available as part of Abacus.ai’s Enterprise offering. + +More on this: + +#### [Anthropic CEO Backs California AI Bill, But Still Has Concerns](https://www.pcmag.com/news/anthropic-ceo-backs-california-ai-bill-but-still-has-concerns) +![](https://i.pcmag.com/imagery/articles/03b85I3BA23x1ZUWUHJwU3F-1.fit_lim.size_1200x630.v1724412145.jpg) + +Anthropic, an AI company, has expressed support for the amended version of California's AI bill SB 1047, despite some reservations. CEO Dario Amodei believes the benefits of the bill, which requires AI companies to adopt and disclose safety and security protocols, outweigh its costs. However, he also notes that the bill could disproportionately affect larger AI companies and potentially lead to overreach by the Attorney General. The bill, which has faced opposition from tech giants like Google and Meta over concerns of stifling innovation, is set for a final vote in the Assembly by August 31. If passed, it will be sent to Governor Gavin Newsom for approval or veto by September 30. + + +More on this: + * [OpenAI’s opposition to California’s AI bill ‘makes no sense,’ says state senator](https://techcrunch.com/2024/08/21/openais-opposition-to-californias-ai-law-makes-no-sense-says-state-senator/) + * [Elon Musk unexpectedly offers support for California’s AI bill](https://techcrunch.com/2024/08/26/elon-musk-unexpectedly-offers-support-for-californias-ai-bill/) + +#### [Ars Technica content is now available in OpenAI services](https://arstechnica.com/information-technology/2024/08/openai-signs-ai-deal-with-conde-nast/) +![](https://cdn.arstechnica.net/wp-content/uploads/2024/08/openai_conde_header_1-760x380.jpg) + +OpenAI has announced a partnership with Condé Nast, the parent company of Ars Technica, to display content from its publications within AI products such as ChatGPT and SearchGPT. This partnership will allow users of these AI services to access information from Condé Nast publications, and will also enable OpenAI to use this content to train future AI language models. The training process, which is computationally intense and expensive, involves feeding content into an AI model's neural network to improve its ability to process conceptual relationships. Despite this partnership, Condé Nast's internal policy still prohibits the use of text generated by AI in its publications. The deal is seen as a strategic move by Condé Nast to expand its content reach, adapt to changing audience behaviors, and ensure proper compensation and attribution for its intellectual property. + + +More on this: + + + +### Other News +#### Tools +![](https://hotshot.co/apple-touch-icon.png) + +[How a 4 Person Team Built Sora](https://hotshot.co/release) - A 4 person team built Sora, a large-scale diffusion transformer model that excels in prompt alignment, consistency, and motion, and is highly extensible to longer durations, higher resolutions, and additional modalities. + +[OpenAI Launches Fine-Tuning Feature for GPT-4o](https://www.pymnts.com/news/artificial-intelligence/2024/openai-launches-fine-tuning-feature-gpt-4o/) - OpenAI has launched a fine-tuning feature for GPT-4o, allowing developers to increase performance and accuracy for their applications by customizing the model's responses with their own datasets. + +[Google Releases Powerful AI Image Generator You Can Use for Free](https://petapixel.com/2024/08/19/google-releases-powerful-ai-image-generator-you-can-use-for-free-imagen-3/) - Google has released a powerful AI image generator, Imagen 3, for free use in the U.S., which outperforms other models and offers advanced editing options, but raises concerns about the data used for training and copyright issues. + +[Meta Presents Sapiens: Foundation for Human Vision Models](https://www-marktechpost-com.cdn.ampproject.org/v/s/www.marktechpost.com/2024/08/23/meta-presents-sapiens-foundation-for-human-vision-models/) - Large-scale pretraining followed by task-specific fine-tuning has revolutionized language modeling and is now transforming computer vision. Extensive datasets like LAION-5B and JFT-300M enable pre-training beyond traditional benchmarks, expanding visual learning capabilities. + +[Perplexity’s latest update improves code interpreter, charts included](https://www.testingcatalog.com/perplexitys-latest-update-improves-code-interpreter-charts-included/) - Perplexity's latest update improves its code interpreter, allowing for the installation of libraries and display of charts in the results, expanding its use cases. + +[Perplexity adds Flux.1 model for Pro users alongside Playground v3 update](https://www.testingcatalog.com/perplexity-adds-flux-1-model-for-pro-users-alongside-playground-v3-update/) - Perplexity introduces new image generation models for Pro users, including the Flux version 1 model, alongside an update to Playground v3. + +[Ideogram AI expands its features with v2 model and color palette options](https://www.testingcatalog.com/ideogram-ai-expands-its-features-with-v2-model-and-color-palette-options/) - Ideogram AI has released its new text-to-image model v2.0, offering improved text rendering, color palette options, and five different models to cater to a wider range of AI-produced content. + +[Lightricks’ LTX Studio, the AI Visual Storytelling Platform, Now Open to All](https://www.businesswire.com/news/home/20240820994937/en/Lightricks%E2%80%99-LTX-Studio-the-AI-Visual-Storytelling-Platform-Now-Open-to-All) - Lightricks has announced the public availability of LTX Studio, an AI-driven storyboarding and prototyping platform designed for creative film and marketing professionals, offering real-time generative and editing solutions, enhanced collaboration, character acting, and generation control. + +[McAfee introduces AI deepfake detection software for PCs ](https://cointelegraph.com/news/mcafee-ai-deepfake-detector-lenovo-pcs-launch) - McAfee introduces AI deepfake detection software for PCs, utilizing advanced AI models trained on nearly 200,000 video samples to quickly and privately determine whether a video has been manipulated, all while maintaining user privacy and device performance. + +[Zed Editor Adds Anthropic-Powered AI Features](https://www.webpronews.com/zed-editor-adds-anthropic-powered-ai-features/) - Zed, the text editor taking the development world by storm, has announced new AI features powered by Anthropic’s Claude. Zed is a new text editor written entirely in Rust, benefiting from the speed, security, and other features the language provides. + +#### Business +![](https://techcrunch.com/wp-content/uploads/2024/05/GettyImages-1450216889.jpg?w=1024) + +[Creatopy, which automates ad creation using AI, raises a $10M Series A](https://techcrunch.com/2024/08/22/creatopy-which-automates-ad-creation-using-ai-raises-a-10m-series-a/) - Creatopy, a startup that automates ad creation using AI, has raised a $10 million Series A and now serves over 5,000 brands and agencies, focusing on scaling, personalization, and automation at scale for digital ads. + +[Waymo is now giving over 100,000 paid self-driving rides per week](https://www.teslarati.com/waymo-100000-paid-rides-week/) - Waymo has surpassed a new milestone with its paid driverless ride-sharing, celebrating over 100,000 paid self-driving rides per week and unveiling its next-generation ride-hailing vehicle platform. + +[AI SDR startups are booming. So why are VCs wary?](https://techcrunch.com/2024/08/22/ai-sdr-startups-are-booming-so-why-are-vcs-wary/) - AI SDR startups are experiencing rapid growth and revenue, but venture capitalists are wary of their long-term success due to concerns about market saturation, profit sustainability, and potential competition from established incumbents. + +[Cruise’s robotaxis are coming to the Uber app in 2025](https://techcrunch.com/2024/08/22/cruises-robotaxis-are-coming-to-the-uber-app-in-2025/) - Cruise, General Motors’ self-driving subsidiary, has signed a multi-year partnership with Uber to bring its robotaxis to the ride-hailing platform in 2025, following a safety incident and regulatory challenges. + +[Salesforce Doubles Down On Autonomous Agents With Einstein SDR and Sales Coach](https://www.cxtoday.com/crm/salesforce-doubles-down-on-autonomous-agents-with-einstein-sdr-and-sales-coach/) - Salesforce introduces Einstein SDR and Sales Coach agents, which autonomously engage with prospects, nurture the pipeline, provide personalized coaching, and aim to help sales teams accelerate growth. + +[Waymo launches its latest driverless ride-hailing platform](https://www.teslarati.com/waymo-latest-driverless-ride-hailing-platform/) - Waymo launches its 6th-generation driverless ride-hailing vehicles, featuring reduced production costs, increased range, compute power, and fewer sensors. + +[Waymo wants to chauffeur your kids](https://techcrunch.com/2024/08/22/waymo-wants-to-chauffeur-your-kids/) - Waymo is considering a subscription program called "Waymo Teen" that would allow teenagers to hail its cars solo and send pickup and drop-off alerts to their parents. + +[Founder of failed fintech Synapse says he’s raised $11M for new robotics startup](https://techcrunch.com/2024/08/22/founder-of-failed-fintech-synapse-says-hes-raised-11m-for-new-robotics-startup/) - Founder of failed fintech Synapse has raised $11M for new robotics startup, Foundation, aiming to create advanced humanoid robots to address labor shortage and automate GDP through AI and robotics. + +[Tesla is hiring workers for $48 an hour to wear motion-capture suits to train its humanoid robots](https://fortune.com/2024/08/19/tesla-robot-hiring-workers-optimus-training-ai/) - Tesla is hiring workers to wear motion-capture suits and gather movement information to train its humanoid robots, which are designed to automate work in company factories. + +[Perplexity AI to launch ads on search platform by fourth quarter](https://finance.yahoo.com/news/perplexity-ai-launch-ads-search-182450047.html) - Perplexity AI plans to introduce advertising on its AI-powered search platform by the fourth quarter, following a successful fundraising round and partnerships with major publishers. + +[China’s ambitions in humanoid robots in full display at expo](https://www.scmp.com/tech/tech-trends/article/3275609/chinas-own-tesla-optimus-beijings-ambitions-humanoid-robots-full-display-expo) - China showcases its progress and ambitions in humanoid robots at the World Robot Conference, with companies unveiling advanced robots powered by large language models and aiming to dominate the field. + +[Chinese firms bypass US export restrictions on AI chips using AWS cloud](https://www.cio.com/article/3493017/chinese-firms-bypass-us-export-restrictions-on-ai-chips-using-aws-cloud.html) - Chinese firms are using cloud services from American companies to bypass US export restrictions on AI chips, allowing them to access restricted technologies. + +[Asked about SAG-AFTRA's strike for better AI protections, Amazon Games Boss claims AI 'has nothing to do with taking work away' from actors because 'for games, we don't really have acting'](https://www.pcgamer.com/games/asked-about-sag-aftras-strike-for-better-ai-protections-amazon-games-boss-claims-ai-has-nothing-to-do-with-taking-work-away-from-actors-because-for-games-we-dont-really-have-acting/) - Amazon Games CEO claims AI in games has nothing to do with taking work away from actors, as games don't really have acting, and discusses how AI will streamline game development processes and help with localization. + +[Google appoints former Character.AI founder as co-lead of its AI models](https://www.msn.com/en-gb/money/technology/google-appoints-former-character-ai-founder-as-co-lead-of-its-ai-models/ar-AA1phEWZ) - nan + +#### Research +![](http://research.google/gr/static/assets/favicon.ico) + +[Transformers in music recommendation](http://research.google/blog/transformers-in-music-recommendation/) - AI-driven music recommendation systems need to consider the broader context of a user's preferences and activities to provide more accurate and valuable song recommendations. + +[Scaling Cross-Embodied Learning: One Policy for Manipulation, Navigation, Locomotion and Aviation](https://arxiv.org/abs/2408.11812v1) - A scalable and flexible transformer-based policy called CrossFormer is proposed to train a single policy across various robot embodiments, demonstrating its ability to control vastly different robots and outperform prior state-of-the-art methods in cross-embodiment learning. + +[Transfusion: Predict the Next Token and Diffuse Images with One Multi-Modal Model](https://arxiv.org/abs/2408.11039v1) - A new multi-modal model, Transfusion, combines language modeling and diffusion to train a single transformer over mixed-modality sequences, scaling significantly better than quantizing images and training a language model over discrete image tokens. + +[Transfusion: Predict the Next Token and Diffuse Images with One Multi-Modal Model](https://arxiv.org/abs/2408.11039v1) - A new multi-modal model called Transfusion combines language modeling and diffusion to train a single transformer over mixed-modality sequences, scaling significantly better than quantizing images and training a language model over discrete image tokens. + +[LongVILA: Scaling Long-Context Visual Language Models for Long Videos](https://arxiv.org/abs/2408.10188v3) - Scaling Long-Context Visual Language Models for Long Videos through the introduction of LongVILA, a full-stack solution for long-context vision-language models, including system, model training, and dataset development. + +[To Code, or Not To Code? Exploring Impact of Code in Pre-training](https://arxiv.org/abs/2408.10914v1) - Including code in pre-training data significantly improves general language model performance across a wide range of tasks, beyond just coding-related ones. + +[Transformers to SSMs: Distilling Quadratic Knowledge to Subquadratic Models](https://arxiv.org/abs/2408.10189v1) - Transformers can be distilled into subquadratic state space models (SSMs) using a method called MOHAWK, allowing SSMs to benefit from the computational resources invested in training Transformer-based architectures. + +[Segment Anything with Multiple Modalities](https://arxiv.org/abs/2408.09085v1) - A new model, MM-SAM, extends the capabilities of the Segment Anything Model (SAM) to support cross-modal and multi-modal processing for robust and enhanced segmentation with different sensor suites. + +[Meta AI Proposes ‘Imagine yourself’: A State-of-the-Art Model for Personalized Image Generation without Subject-Specific Fine-Tuning](https://www-marktechpost-com.cdn.ampproject.org/v/s/www.marktechpost.com/2024/08/21/meta-ai-proposes-imagine-yourself-a-state-of-the-art-model-for-personalized-image-generation-without-subject-specific-fine-tuning/) - Personalized image generation is gaining traction due to its potential in various applications, from social media to virtual reality. However, traditional methods often require extensive tuning for each user, limiting efficiency and scalability. + +[TableBench: A Comprehensive and Complex Benchmark for Table Question Answering](https://arxiv.org/abs/2408.09174v1) - Advancements in Large Language Models have improved the processing of tabular data, leading to the creation of a comprehensive benchmark called TableBench to address the challenges of applying LLMs in industrial scenarios. + +[Training-free Graph Neural Networks and the Power of Labels as Features](https://arxiv.org/abs/2404.19288v2) - Training-free Graph Neural Networks can leverage the power of labels as features, eliminating the need for extensive training. + +[Automating Thought of Search: A Journey Towards Soundness and Completeness](https://arxiv.org/abs/2408.11326v1) - Automating Thought of Search (ToS) is a method that uses language models to define the search space with code, and a recent work has proposed automating ToS (AutoToS) to completely remove the human from solving planning problems, achieving 100% accuracy. + +#### Concerns +![](https://cdn.vox-cdn.com/thumbor/1JcCLf5KFOjJLaFLNDqKDi0kfyw=/0x0:4000x2662/1200x628/filters:focal(1948x1203:1949x1204)/cdn.vox-cdn.com/uploads/chorus_asset/file/25584525/2166080531.jpg) + +[Google DeepMind staff call for end to military contracts](https://www.theverge.com/2024/8/22/24226161/google-deepmind-staff-call-for-end-to-military-contracts) - Google DeepMind employees call for an end to military contracts due to concerns about the use of AI technology for warfare, urging the company to investigate and cut off military access to their technology. + +[Google’s AI ‘Reimagine’ tool helped us add wrecks, disasters, and corpses to our photos](https://www.theverge.com/2024/8/21/24224084/google-pixel-9-reimagine-ai-photos) - Google's new AI photo editing tool, Reimagine, allows users to add realistic and disturbing elements to their photos, raising concerns about the potential for misuse and the lack of safeguards to identify manipulated content. + +[Authors sue Claude AI chatbot creator Anthropic for copyright infringement](https://abcnews.go.com/US/wireStory/authors-sue-claude-ai-chatbot-creator-anthropic-copyright-112964872) - Authors are suing AI startup Anthropic for training its chatbot Claude on pirated copies of copyrighted books, alleging "large-scale theft" and challenging the company's claim of responsible and safety-focused AI development. + +#### Policy +![](https://gizmodo.com/app/uploads/2024/08/joe-biden-dnc-aug-19-2024.jpg) + +[Fake Biden Robocalls Cost Wireless Provider $1 Million in FCC Penalties](https://gizmodo.com/fake-biden-robocalls-cost-wireless-provider-1-million-in-fcc-penalties-2000489648) - Fake Biden robocalls cost wireless provider $1 million in FCC penalties after allowing deepfake calls to be transmitted during New Hampshire primaries, leading to a settlement and new network security measures. + +#### Analysis +![](https://wp.technologyreview.com/wp-content/uploads/2024/08/OpenSourceAI.jpg?resize=1200,600) + +[We finally have a definition for open-source AI](https://www.technologyreview.com/2024/08/22/1097224/we-finally-have-a-definition-for-open-source-ai/) - Open-source AI is defined as a system that can be used, inspected, modified, and shared without restrictions, addressing the lack of a standard definition and potential misuse of the term by companies. + +
+ +Copyright © 2024 Skynet Today, All rights reserved. diff --git a/assets/img/digests/285/03b85I3BA23x1ZUWUHJwU3F-1.fit_lim.size_1200x630.v1724412145.jpg b/assets/img/digests/285/03b85I3BA23x1ZUWUHJwU3F-1.fit_lim.size_1200x630.v1724412145.jpg new file mode 100644 index 00000000..81a5233e Binary files /dev/null and b/assets/img/digests/285/03b85I3BA23x1ZUWUHJwU3F-1.fit_lim.size_1200x630.v1724412145.jpg differ diff --git a/scripts/csv2podcast_notes.py b/scripts/csv2podcast_notes.py index 377d6cb4..a1a2c3b9 100644 --- a/scripts/csv2podcast_notes.py +++ b/scripts/csv2podcast_notes.py @@ -62,7 +62,7 @@ def summarize_article(url, lighting_round_story=False, save_image=False): im_response = requests.get(article.top_image) if im_response.status_code == 200: im_name = article.top_image.split("/")[-1] - with open(im_name, "wb") as f: + with open("images/"+im_name, "wb") as f: f.write(im_response.content) system_prompt = '''