Big Data Analytics: Systems, Algorithms, Applications

ISBN-13: 9789811500930 / Angielski / Twarda / 2019 / 412 str.

C. S. R. Prabhu; Aneesh Sreevallabh Chivukula; Aditya Mogadala

Spis treści

Table of contents:

Preface

 

Chapter 1 Big Data

1.1       Introduction

1.2       What is Big Data?

1.3       Disruptive change and paradigm shift and its business meaning

1.4       HADOOP

1.4.1    Silos

1.4.2    Big Bang of Big Data

1.4.3    Possibilities

1.4.4    Future

1.4.5    Parallel processing for problem solving

1.4.6    Why Hadoop?

1.4.7    Hadoop and HDFS

1.4.8    Hadoop Version 1.0 & 2.0

     1.4.8.1 Limitations of Hadoop 1.0

1.4.9    Hadoop 2.0

1.5       HDFS Overview

1.5.1    Map Reduce framework

1.5.2    Job Tracker

1.5.3    YARN

1.6       Hadoop Eco System

1.6.1    Cloud based Hadoop Solutions

1.6.2    SPARK and Data Stream Processing                   

1.7       Decision Making and Data Analysis in the Context of Big Data Environment

1.7.1    Present Day Data Analytics Techniques

1.8       Machine Learning Algorithms

1.9       Evolutionary Computing (EC)

Conclusion

Review Questions

References

 

Chapter 2: Intelligent Systems

2.1       Introduction

2.2       Machine Learning Paradigms

2.2.1    Open Source Data Science

2.2.2    Machine Intelligence and Computational Intelligence

2.2.3    Data Engineering and Data Sciences

2.3       Machine Learning Paradigms

2.4       Big Data Computing

2.4.1    Distributed Systems and Database Systems

2.4.2    Data Stream Systems and Stream Mining

2.4.3    Ubiquitous Computing Infrastructures

Conclusion

Review Questions

References

 

Chapter 3: Predictive Modeling for Unstructured Data

3.1       Introduction

3.2       Applications of Predictive Modeling

3.3       Feature Engineering

3.4       Pattern Mining for Predictive Modeling

Conclusion

Review Questions

References

 

Chapter 4: Machine Learning Algorithms for Big Data

4.1 Introduction

4.2 Generative vs Discriminative Algorithms

4.3 Supervised Learning for Big Data

4.3.1 Decision Trees

4.3.2 Logistic Regression

4.3.3  Regression and Forecasting

4.3.4 Supervised Neural Networks

4.3.5 Support Vector Machines

4.4 Unsupervised Learning for Big Data

4.4.1 Spectral Clustering

4.4.2 Principal Component Analysis (PCA)

4.4.3 Latent Dirichlet Allocation (LDA)

4.4.4 Matrix Factorization

4.4.5 Manifold Learning

4.5 Semi-Supervised Learning for Big Data

4.5.1 Co-training

4.5.2 Label Propagation

4.5.3 Multi-View Learning

4.6 Reinforcement Learning Basics for Big Data

4.6.1 Markov Decision Process

4.6.2 Planning

4.6.3 RL in practice

4.7 Online Learning for Big Data

Conclusion

Review Questions

References

 

Chapter 5: Analytics Models for Data Science

5.1       Introduction

5.2       Data Models

5.2.1    Data Products

5.2.2    Data Munging

5.2.3    Descriptive Analytics

5.2.4    Predictive Analytics

5.2.5    Data Science

5.2.6    Network Science

5.3       Computing Models

5.3.1    Data Structures for Big Data

5.3.2    Feature Engineering for Structured Data

            5.3.2.1 Feature Construction

            5.3.2.2 Feature Extraction

            5.3.2.3 Feature Selection

            5.3.2.4 Feature Learning

            5.3.2.5 Ensemble Learning

5.3.3    Computational Algorithmics

5.4       Programming Models

5.4.1 Parallel Programming

5.4.2 Functional Programming

5.4.3 Distributed Programming

Conclusion

Review Questions

References

 

Chapter 6: Social Semantic Web Mining and Big Data Analytics

6.1       Introduction

6.2       What is Semantic Web?

6.3       Knowledge Representation Techniques and Platforms in Semantic Web

6.4       Web Ontology Language (OWL)

6.5       Object Knowledge Model (OKM)

6.6       Ontology

6.7       Architecture of Semantic Web and the Semantic Web Road Map

6.8       Social Semantic Web Mining

6.9       Conceptual Networks and Folksonomies or Folk Taxonomies of Concepts/Sub Concepts

6.10     SNA and ABM

6.11     e-Social Science

6.12     Opinion Mining and Sentiment Analysis

Conclusion

Review Questions

References

 

Chapter 7: Internet of Things (IoT) and Big Data Analytics

7.1       Introduction

7.2       Smart Cities and IOT Sectoral Applications

7.3       Stages of IOT And Stakeholders

7.3.1    Stages of IOT

7.3.2    Stakeholders

7.3.3    Practical Down Scaling

7.4       Analytics

7.4.1    Analytics from the Edge to Cloud

7.5       Access

7.6       Cost Reduction

7.7       Opportunities and Business Model

7.8       Content and Semantics

7.9       Data base Business models coming out of IOT

7.10     Future of IOT

            7.10.1 Technology Drivers

            7.10.2  Future possibilities

7.10.3 Challenges and Concerns

7.11     Big Data Analytics and IOT

7.11.1 Infrastructure for integration of Big Data with IoT

7.12     Fog Computing and Fog Analytics

7.13     Research Trends

Conclusion

Review Questions

References

 

Chapter 8: Big Data Analytics for Financial Services and Banking

8.1       Introduction

8.2       Customer Insights and Marketing Analysis

8.3       Sentiment Analysis for consolidating customer feedback.

8.4       Predictive Analytics for capitalizing on customer insights

8.5       Model building

8.6       Fraud detection and Risk management

8.7       Integration of Big Data Analytics into operations.

8.8       How banks can benefit from Big Data Analytics?

8.9       Best practices of Data Analytics in banking for crises redressal and management

8.10     Bottlenecks

Conclusion

Review Questions

References

 

Chapter 9: Big Data Analytics Techniques in Capital Market Use Cases

9.1       Introduction

9.2       Capital Market Use Cases of Big Data Technologies

9.2.1    Algorithmic Trading

9.2.2    Investors’ Faster Access to Securities

9.3       Prediction Algorithms

9.3.1    Stock Market Prediction

9.3.2    Efficient Market Hypothesis

9.3.3    Random Walk Theory (RWT)

9.3.4    Trading Philosophies

9.3.5    Simulation Techniques

9.4       Research Experiments to determine threshold time for determining predictability

9.5       Experimental Analysis using Bag of Words and Support Vector Machine (SVM) Application to News Articles

9.6       Textual Representation and Analysis of News Articles

9.7       Named Entities

9.8       Object Knowledge Model (OKM)

9.9       Application of Machine Learning Algorithms

9.9.1    Sources of Data

9.10     Future Work

Conclusion

Review Questions

References

 

Chapter 10: Big Data Analytics for Insurance

10.1     Introduction

10.2     The Insurance Business Scenario

10.3     Big Data Deployment in Insurance

10.4     Insurance Use Cases

10.5     Customer Needs Analysis

10.6     Other Applications

Conclusion

Review Questions

References

 

Chapter 11: Security in Big Data

11.1     Introduction

11.2     Ills of Social Networking – Identity Theft

11.3     Organizational Big Data Security

11.4     Security in Hadoop

11.5     Issues and Challenges in Big Data Security

11.6     Encryption for Security

11.7     Secure Map -Reduce and Log Management

11.8     Access Control, Differential Privacy and Third Party Authentication

11.9     Real Time Access Control

11.10   Security Best Practices for Non-Relational Or NoSQL Databases                                     11.11      Challenges, Issues and New Approaches

11.12   Research Overview and New Approaches for Security Issues in Big Data 

Conclusion

Review Questions

References

 

Chapter 12: Privacy and Big Data Analytics

12.1     Introduction

12.2     Privacy Protection

12.3     Enterprise Big Data Privacy Policy And COBIT

12.4     Assurance and Governance

12.5     Challenges, Issues and Approaches For Privacy Protection

Conclusion

Review Questions

References

 

Chapter 13: Emerging Trends in Big Data

13.1 Learning to Generate

            13.1.1 Unstructured Data

            13.1.2 Structured Data

            13.1.3 Multi-agent Environments

13.2 Learning to Learn

            13.2.1 Model Selection

            13.2.2 Bayesian Optimization

            13.2.3 Transfer Learning

13.3 Adversarial Learning

            13.3.1 Anomaly Detection

            13.3.2 Concept Drift

            13.3.3 Adversarial Networks

13.4 Complex Networks

            13.3.1 Graph Databases

            13.3.2 Knowledge Graphs

            13.3.3 Event Mining

            13.3.3 Dynamic Network Analysis

Conclusion

Review Questions

References

 

Case Studies

1.      Google

2.      General Electric

3.      Microsoft

4.      Nokia

5.      Facebook

6.      OPower

7.      Kaggle

8.      Deutsche Bank

9.      Health Sector Analysis

10.  Online Insurance

11.  Delta Airlines

12.  Linked.In

13.  Traffic Management

14.  CISCO

 

Appendix 1: Databases for Big Data: NoSQL Databases Column Databases and Graph Databases

Appendix 2:  HDFS and MapReduce

Appendix 3:  Statistical Foundations

Appendix 4:  Probability, Random variables and mathematical expectation

Appendix 5:  R Language Features and Its Applications in Machine Learning (Clustering,

                       Classification and Regression)

Appendix 6: Spark and Scala for data streams

O autorze

Dr. Chivukula Sree Rama Prabhu has held prestigious positions with Government of India and various institutions. He retired as Director General of the National Informatics Centre (NIC), Ministry of Electronics and Information Technology, Government of India, New Delhi, and has worked with Tata Consultancy Services (TCS), CMC, TES and TELCO (now Tata Motors). He was also faculty for the Programs of the APO (Asian Productivity Organization).  He has taught and researched at the University of Central Florida, Orlando, USA, and also had a brief stint as a Consultant to NASA. He was Chairman of the Computer Society of India (CSI), Hyderabad Chapter. He is presently working as an Advisor (Honorary) at KL University, Vijayawada, Andhra Pradesh, and as a Director of Research and Innovation at Keshav Memorial Institute of Technology (KMIT), Hyderabad.

He received his Master’s degree in Electrical Engineering with specialization in Computer Science from the Indian Institute of Technology, Bombay. He has guided many Master’s and doctoral students in research areas such as Big Data.

Dr. Aneesh Sreevallabh Chivukula is currently a Research Scholar at the Advanced Analytics Institute, University of Technology Sydney (UTS), Australia. Previously, he chiefly worked in computational data science-driven product development at Indian startup companies and research labs. He received his M.S. degree from the International Institute of Information Technology (IIIT), Hyderabad. His research interests include machine learning, data mining, pattern recognition, big data analytics and cloud computing.

Dr. Aditya Mogadala is a postdoc in the Language Science and Technology at Saarland University. His research concentrates on the general area of Deep/Representation learning applied for integration of external real-world/common-sense knowledge (e.g., vision and knowledge graphs) into natural language sequence generation models. Before Postdoc, he was a PhD student and Research Associate at the Karlsruhe Institute of Technology, Germany. He holds B.Tech and M.S. degree from the IIIT, Hyderabad, and has worked as a Software Engineer at IBM India Software Labs.

Mr. Rohit Ghosh currently works at Qure, Mumbai.  He previously served as a Data Scientist for ListUp, and for Data Science Labs. Holding a B.Tech. from the IIT Mumbai, his work involves R&D areas in computer vision, deep learning, reinforcement learning (mostly related to trading strategies) and cryptocurrencies.

Dr. Jenila Livingston is an Associate Professor with the CSE Dept at VIT, Chennai. Her teaching foci and research interests include artificial intelligence, soft computing, and analytics.

Blurb

This book provides a comprehensive survey of techniques, technologies and applications of Big Data and its analysis. The Big Data phenomenon is increasingly impacting all sectors of business and industry, producing an emerging new information ecosystem. On the applications front, the book offers detailed descriptions of various application areas for Big Data Analytics in the important domains of Social Semantic Web Mining, Banking and Financial Services, Capital Markets, Insurance, Advertisement, Recommendation Systems, Bio-Informatics, the IoT and Fog Computing, before delving into issues of security and privacy. 

With regard to machine learning techniques, the book presents all the standard algorithms for learning – including supervised, semi-supervised and unsupervised techniques such as clustering and reinforcement learning techniques to perform collective Deep Learning. Multi-layered and nonlinear learning for Big Data are also covered. 

In turn, the book highlights real-life case studies on successful implementations of Big Data Analytics at large IT companies such as Google, Facebook, LinkedIn and Microsoft. Multi-sectorial case studies on domain-based companies such as Deutsche Bank, the power provider Opower, Delta Airlines and a Chinese City Transportation application represent a valuable addition. 

Given its comprehensive coverage of Big Data Analytics, the book offers a unique resource for undergraduate and graduate students, researchers, educators and IT professionals alike.