Tweetypie

Core Tweet service for reading and writing Tweet data

Overview

Tweetypie is the core Tweet service that handles the reading and writing of Tweet data. It is called by:

• Twitter clients (through GraphQL)
• Internal Twitter services

Tweetypie provides endpoints to fetch, create, delete, and edit Tweets, and calls several backend services to hydrate Tweet-related data.

Architecture

Tweetypie acts as the central orchestration layer for Tweet operations:

Twitter Clients (GraphQL) / Internal Services
                ↓
            Tweetypie
                ↓
    ┌───────────┼───────────┬──────────┬─────────┐
    ↓           ↓           ↓          ↓         ↓
Manhattan  Twemcache   Talon    MediaService  Other
  (Storage)  (Cache)  (URLs)    (Media)     Backends

Read Path

The read path fetches Tweet data from storage/cache and hydrates it with data from various backend services.

1. Request Handling

   The get_tweets request is handled by GetTweetsHandler.

2. Tweet Retrieval

   TweetResultRepository fetches Tweet data:

   • First checks Twemcache (distributed cache)
   • Falls back to Manhattan (distributed database) if cache miss
3. Hydration Pipeline

   Raw Tweet data passes through the hydration pipeline to enrich it with:

   • Expanded URLs
   • Media metadata
   • User mentions
   • Cards and other entities
4. Response

   Fully hydrated Tweet is returned to the caller

Relevant Packages

Backends

Wrappers around Thrift services that Tweetypie calls:

// Example: Talon backend for URL shortening
object Talon extends Backend {
  def expandUrls(shortUrls: Seq[String]): Future[Seq[ExpandedUrl]]
}

Backends are located at: tweetypie/server/src/main/scala/com/twitter/tweetypie/backends/

Repositories

Provide structured interfaces for retrieving data from backends:

// UrlRepository wraps the Talon backend
class UrlRepository(talon: TalonBackend) {
  def getExpandedUrls(tweetId: Long): Future[Seq[Url]] = {
    // Fetch and format URL data
  }
}

Repositories are located at: tweetypie/server/src/main/scala/com/twitter/tweetypie/repository/

Hydrators

Enrich raw Tweet data with additional information:

// UrlEntityHydrator expands t.co links
class UrlEntityHydrator(urlRepository: UrlRepository) extends Hydrator {
  def hydrate(tweet: Tweet): Future[Tweet] = {
    urlRepository.getExpandedUrls(tweet.id).map { expandedUrls =>
      tweet.copy(urls = expandedUrls)
    }
  }
}

Hydrators fetch data using repositories and attach it to Tweets with metadata indicating hydration success.

Hydrators are located at: tweetypie/server/src/main/scala/com/twitter/tweetypie/hydrator/

Handlers

Functions that handle requests to Tweetypie endpoints:

// GetTweetsHandler processes get_tweets requests
class GetTweetsHandler(tweetRepo: TweetResultRepository) {
  def apply(request: GetTweetsRequest): Future[Seq[Tweet]] = {
    tweetRepo.get(request.tweetIds, request.options)
  }
}

Handlers are located at: tweetypie/server/src/main/scala/com/twitter/tweetypie/handler/

Through the Read Path

Detailed flow of a get_tweets request:

1. GetTweetsHandler receives the request
2. Uses TweetResultRepository (defined in LogicalRepositories.scala:301)
3. TweetResultRepository uses:
   • ManhattanTweetRepository - fetches from Manhattan storage
   • Wrapped in CachingTweetRepository - adds Twemcache caching layer
   • Wrapped in hydration layer - applies all hydrators
4. Raw Tweet data flows through TweetHydration pipeline
5. Fully hydrated Tweet returned to caller

The hydration pipeline is described in: tweetypie/server/src/main/scala/com/twitter/tweetypie/hydrator/TweetHydration.scala:789

Write Path

The write path creates or modifies Tweets and updates various backend stores.

1. Request Handling

   The post_tweet request is handled by PostTweet.scala.

2. Tweet Building

   TweetBuilder creates a Tweet from the request:

   • Text processing and validation
   • URL shortening via Talon
   • Media processing
   • Duplicate detection
3. Write Path Hydration

   WritePathHydration.hydrateInsertTweet hydrates the Tweet before storage to ensure all required fields are populated.

4. Store Updates

   Tweet data is written to various stores as described in InsertTweet.scala:

   • Manhattan (primary storage)
   • Timeline Service (for timeline fanout)
   • Search Index (for tweet search)
   • Other downstream services

Relevant Packages

Stores

Define logic for updating backends on write:

// ManhattanTweetStore writes Tweets to Manhattan
object ManhattanTweetStore extends Store {
  def insertTweet(tweet: Tweet): Future[Unit] = {
    manhattanClient.insert(
      key = tweet.id,
      value = serialize(tweet)
    )
  }
}

Stores are located at: tweetypie/server/src/main/scala/com/twitter/tweetypie/store/

Store Modules

Define logic for handling write endpoints and coordinate which stores to call:

// InsertTweet handles post_tweet endpoint
object InsertTweet extends StoreModule {
  // Defines which stores are called for insert operations
  val stores = Seq(
    ManhattanTweetStore,
    TimelineStore,
    SearchIndexStore,
    // ... other stores
  )
}

Store modules are located at: tweetypie/server/src/main/scala/com/twitter/tweetypie/store/InsertTweet.scala:84

Through the Write Path

Detailed flow of a post_tweet request:

1. PostTweet.scala handles the request (line 338)
2. TweetBuilder creates Tweet:
   • Validates text and media
   • Shortens URLs via Talon
   • Processes media uploads
   • Checks for duplicates
3. WritePathHydration.hydrateInsertTweet enriches Tweet (WritePathHydration.scala:54)
4. InsertTweet module writes to stores (InsertTweet.scala:84):
   • Manhattan (primary storage)
   • Cache (Twemcache)
   • Timeline Service
   • Search Index
   • TFlock (for fanout)
   • EventBus (for streaming)

Key Operations

Creating Tweets

// Post a new tweet
val postTweetRequest = PostTweetRequest(
  userId = userId,
  text = "Hello, world!",
  mediaUploadIds = Seq(mediaId),
  placeId = Some(placeId)
)

val tweet = tweetypie.postTweet(postTweetRequest)

Reading Tweets

// Fetch tweets with hydration options
val getTweetsRequest = GetTweetsRequest(
  tweetIds = Seq(tweetId1, tweetId2),
  options = GetTweetsOptions(
    includeCards = true,
    includeMedia = true,
    includeUser = true,
    safetyLevel = SafetyLevel.Recommendations
  )
)

val tweets = tweetypie.getTweets(getTweetsRequest)

Deleting Tweets

// Delete a tweet
val deleteTweetRequest = DeleteTweetRequest(
  tweetId = tweetId,
  userId = userId,
  auditNote = Some("User requested deletion")
)

tweetypie.deleteTweet(deleteTweetRequest)

Editing Tweets

// Edit an existing tweet
val editTweetRequest = EditTweetRequest(
  tweetId = tweetId,
  userId = userId,
  newText = "Updated tweet text"
)

val editedTweet = tweetypie.editTweet(editTweetRequest)

Data Storage

Manhattan

Twitter’s distributed key-value store:

• Primary storage for Tweet data
• Highly available and scalable
• Optimized for low-latency reads

Twemcache

Twitter’s distributed caching layer:

• Caching frequently accessed Tweets
• Reduces load on Manhattan
• Maintains consistency with write-through pattern

Hydration Details

Tweetypie hydrates various Tweet components:

URLs

Expand t.co shortened URLs via Talon

Media

Fetch media metadata and thumbnails

Mentions

Hydrate user mentions with profile data

Cards

Fetch Twitter Card metadata

Places

Hydrate location/place information

Quotes

Fetch quoted Tweet data

Hydration Pipeline

// Simplified hydration pipeline
val hydrationPipeline = Seq(
  UrlEntityHydrator,
  MediaEntityHydrator,
  MentionEntityHydrator,
  CardHydrator,
  QuotedTweetHydrator,
  ConversationControlHydrator,
  // ... many more hydrators
)

val hydratedTweet = hydrationPipeline.foldLeft(rawTweet) { (tweet, hydrator) =>
  hydrator.hydrate(tweet)
}

Source: tweetypie/server/src/main/scala/com/twitter/tweetypie/hydrator/TweetHydration.scala

Safety and Visibility

Tweetypie enforces various safety and visibility rules:

Visibility Filtering

• Blocked/muted users
• Protected accounts
• NSFW content filtering
• Age-gated content
• Country-specific takedowns

Safety Levels

Different safety levels for different contexts:

• TimelineHome - Strictest filtering for Home timeline
• Recommendations - Balanced for recommendation surfaces
• Search - Search-appropriate filtering
• Minimal - Minimal filtering for moderation tools

Performance Optimization

Caching Strategy

// Multi-layer caching
val tweet = cache.get(tweetId) match {
  case Some(cached) => cached
  case None => 
    val fresh = manhattan.get(tweetId)
    cache.set(tweetId, fresh)
    fresh
}

Batch Operations

// Batch get tweets for efficiency
val tweets = tweetypie.getTweets(
  tweetIds = (1 to 100).map(_.toLong),
  options = GetTweetsOptions(...)
)

Async Hydration

Some hydrators run asynchronously to reduce latency:

• Non-critical data fetched in background
• Progressive hydration for incremental responses
• Parallel hydration where possible

Tweetypie is in the critical path for most Twitter operations. p99 latency must stay below 100ms for read operations.

Monitoring

Key Metrics

• Request Rate: Requests per second by endpoint
• Latency: p50, p99, p999 for reads and writes
• Success Rate: Non-error responses
• Cache Hit Rate: Twemcache effectiveness
• Hydration Success: Per-hydrator success rates

Alerts

• High latency (p99 > 200ms)
• Low success rate (< 99.9%)
• Cache performance degradation
• Backend service failures
• Data consistency issues

Related Services

• Home Mixer - Primary consumer for timeline tweets
• Timeline Ranker - Uses Tweetypie for tweet hydration
• Pushservice - Fetches tweet data for notifications
• CR Mixer - Uses tweet data for candidate generation