Singapore
United States
India
Scaling Hadoop up to the Cloud
The advent of Hadoop for processing Big Data many years ago solved the problem of needing to scale hardware to specialized and impractical dimensions, both from the specification as well as cost points of view. Hadoop distributes processing and storage to smaller units of hardware and allows new hardware to be added as required. These smaller units of hardware could be cheap, non-specialized commodity server hardware. This makes the proposition of working with Big Data more attractive from the point of view of the investment [...]
Adoption of Telematics in Auto Insurance
Telematics is the application of a combination of technologies to gather data from automobiles and transmit it to a receiving organization for various purposes. It could also be two-way communication, and this again depends on the purpose. The use of telematics is an example of an end point technology that forms one part of the larger world of IoT (Internet of Things), but the concept of transmitting data to and from a vehicle actually came into existence many decades ago. It was first pioneered by [...]
Blockchain Technology: The Next Disruption in the BFSI Sector?
much interest. Why is it growing? In almost every financial transaction there is an element of trust involved. Before a seller hands over a good or service in exchange for money they need to be trust that the buyer has enough money to pay for the purchase, the same money is not simultaneously being committed for spending elsewhere, and that the payment will be duly made at the agreed time and mode. In the simplest form it could be a small cash transaction in a [...]
Traits of a Great Data Scientist
Data scientists are a rare breed. It’s not easy to find the perfect blend of skills in information technology, statistics and business. Having said that, more and more are slowly beginning to be emerge after building up their profiles through a mix of experience and training. We’re still a long way off, however, in terms of bridging the gap between supply and demand in the talent market. Data scientists think data. They look for it, have enough programming knowledge to extract it, know how to [...]
Integration of Predictive Analytics into the Enterprise
A couple of decades ago I had the opportunity to work on a business problem that initially seemed rather daunting. The challenge was to optimize the operations of a precision engineering goods manufacturer by moving to a completely different approach to planning. Our company initially used to follow the practices that were derived from the management thinking that worked with the economic environment of the 70s, when the question of how to always have stock of whatever our customers could want was answered by always [...]
Risk Management in Analytics Projects
Business analytics with big data is increasingly being used to help manage risk in a number of areas such as operations, finance, insurance and security. While the broad lifecycle phases of any analytics project remain pretty much the same, the techniques that are used to implement each phase may vary a little depending on the type of data, its sources, etc. The amount of time and effort invested in each phase may also vary greatly across projects. Increased variability in method, time and effort [...]
The Art and Science of Data Visualization
Many companies have succeeded in implementing analytics and using it for various purposes. Some of the better known companies that were early adopters included those like GE, a company that has always been associated with excellence and innovation driven by facts and metrics programs (commonly known these days as a data driven culture). They are huge believers in the power of data, and use it to improve every aspect of the way they work, and also to improve their products. Of course, there are many [...]
Analytics and the Role of the CEO
As the adoption of business analytics continues to grow, greater clarity is being gained in understanding what goes into achieving results with it, and how much effort various functions need to contribute to it. In earlier posts I’ve expressed my perspective on howanalytics projects should be run, and what factors make them successful and productive. Working with analytics requires a lot of technical skill to source, extract and integrate the right data. After that, the right statistical techniques need to be applied, and strong business [...]
Does Automated Analytics need Manual Intervention?
As the adoption of predictive analytics in business continues to pick up, there is also a parallel thrust to harness the use of Big Data for analytics purposes. Working with Big Data, of course, means working with huge volumes of varied data. The statistical techniques that are used in analytics are not new, but their application across a much larger number of purposes and across more functions and sectors of industry has boomed only over the past few years. This has been made possible because [...]
Dipping in the Data Lake
Over the past couple of years the term data lake has increasingly come into prominence. It was first used by James Dixon, the Chief Technology Officer of Pentaho back in 2010, and more recently the concept has been hyped up a lot by a number of technology product vendors to promote their offerings in this space. It’s well known by now that although most enterprises collect and store a lot of data in warehouses a lot of data is not collected at all, and certainly [...]