Intro to Stream for Continuous Data Processing

Okay, so the next topic we'll be talking about is stream, which is also a very essential part of the core node.js. So here's the problem with large data. First of all, the speed is dismal because you have to wait for the entire large data resource to load. Then you would hit a buffer limit at around one gigabyte. So what is streams?

Streams, it's an abstraction for continuous chunking of data. So you can start processing your data right from the first chunk, without having to wait for the entire data to load. And it's very, very, very more performant than the traditional approach.

There are four categories of streams. Readable, you might guess it's a stream which you can read from. Writable, you would write to that stream. Duplex, you can use both as readable and writable. And transform, which, as you can guess from the name, will allow you to transform the data. So streams inherit from event emitters.

That's why first we've covered event emitters, so now you can see the actual implementation of the event emitters with actual usage in real life when you work with streams. Also, streams are pretty much everywhere in node.js, especially in the core node.js. So it's in HTTP request and responses, it's in standard input and output when you're working your system. And then, when you read or write files, you can also use streams.

So let's take a look at the readable stream. Readable stream, it's when you're getting some information and you want to start processing it right away. So there is a read operation. And in the demo, in the process.stdin, that's your input from the terminal. So this is the code, stdin.js. First two lines, we just need to do that for stream to work. And then the actual way we'll capture the data, it's happening in the event listener, onData. So onData, we're getting a chunk in the callback. The name doesn't matter, just the first argument. And then when we finish, when we've exhausted all the data, when the stream is over, we would get the .onEnd event placeholder. So the demo, just go to the source code, node stdin.js.