August 1, 2021
Welcome to my Cliche…
The advent of Big data no doubt has proliferated the need for Data Engineers.
But what is Big data?
They are data with 3 V’s and can’t be processed by traditional method.
The 3 V’s are:
Volume(large like Gigabytes,Pb..), Velocity (frequent, stream, real-time…) Variety ( different formats like csv, parquet, audio, video..) This kind of data needs specialized infrastructures, Schema, Storage, Processing tools, automation & monitoring.