Flume is responsible for producing data and sending it to kafka.

Spark streaming acts as a consumer, and gets the data from kafka in real time for calculation.

The results are saved to redis for real-time recommendation.

flume+kafka+spark+redis is a classic architecture for real-time data collection and computation.

For more big data related technologies, you can search "lxw's big data field".