We had to rethink our entire approach to message processing from the ground up. Leveraging Kafka as our message broker, we learned some powerful techniques for high-performance message ingestion