background.md

Fortis background information

Overview

Project Fortis is a data ingestion, analysis and visualization pipeline. The Fortis pipeline collects social media conversations and postings from the public web and darknet data sources.

Monitoring

Fortis is a flexible project and can be configured for many situations, e.g.:

• Ingesting from multiple data sources, including:
  • Twitter
  • Facebook
  • Public Web (via Bing)
  • RSS
  • Reddit
  • Instagram
  • Radio Broadcasts
  • ACLED
• Fortis also comes with pre-configured terms to monitor sites of these types:
  • Humanitarian
  • Climate Change
  • Health

Architecture

Articles

• Runtime Configuration of Spark Streaming Jobs
• Spark Streaming Transformations: A Deep-dive
• Permissively-Licensed Named Entity Recognition on the JVM
• Building a Custom Spark Connector for Near Real-Time Speech-to-Text Transcription
• Geocoding Social Conversations with NLP and JavaScript
• Accelerating UN Humanitarian Aid Planning with GraphQL

Videos

• Dashboard walkthrough (in Spanish)
• KubeCon conference talk
• Microsoft Build conference talk
• Spark Summit conference talk

Open Source

We published several generally useful open source projects as part of the development of Fortis, including:

• Spark Streaming connector for Facebook
• Spark Streaming connector for Instagram
• Spark Streaming connector for Reddit
• Spark Streaming connector for Bing
• Spark Streaming connector for RSS
• Advanced filtering options for Twitter Spark Streaming connector