Current location - Loan Platform Complete Network - Big data management - What is the course system of big data in order to Road
What is the course system of big data in order to Road
Big data technology is very widely used today, many want to enter the profession to learn big data training children's shoes do not know where to start, from where to start to learn first according to your basic situation, if you are zero basis also do not need to worry about the first from the foundation to start to learn on it, then learn the basic java to start, data structure, relational databases, linux system operation, consolidate the foundation, then enter the study of big data, such as hadoop offline analysis, Storm real-time computing, spark memory computing study. Then enter the study of big data, for example: hadoop offline analysis, Storm real-time computing, spark memory computing learning, to Road Education big data course system can be as follows:

The first phase of WEB development foundation

HTML foundation

1, Html basic introduction

2, HTML syntax specification

3, basic HTML syntax specification

2, HTML syntax specification

3, HTML syntax specification

3, HTML syntax specification

3, HTML syntax specification

3, the introduction of basic tags

4, HTML editor / text document / WebStrom / elipse

5, HTML elements and attributes

6, the basic HTML elements

6.1 Title

6.2 paragraphs

6.3 styles and style attribute

6.3 Links a

6.4 Images img

6.5 Tables table

6.6 Lists ul/ol/dl

7, HTML annotations

8, Introduction to Forms

9, Table tags

10, Introduction to the DIV layout

11, HTML lists in detail

HTML Layout and Bootstrap

1, HTML block elements (block) and in-line elements (inline)

2, the use of div to achieve the layout of the page

3, responsive WEB design (Responsive Web Design

4, the use of bootstrap to achieve responsive layout

HTML form elements

1, HTML form form

2, HTML form elements

3, HTML input type type

4, Html input attributes

< p>CSS Basics

1, Introduction to CSS and Basic Syntax

2, Using CSS in HTML Documents

3, CSS Styles

4, CSS Selectors

5, Box Model

6, Layout and Positioning

CSS Advanced/CSS3

1, Sizing and alignment

2. Classification (clear/cursor/display/float/position/visibility)

3. Navigation bar

4. Picture gallery

5. Picture transparency

6. Media type @media

7, CSS3

8, CSS3 animation effects

JavaScript Basics

1, Introduction to JavaScript

2, Basic Syntax Rules

3, Using JS in HTML Documents

4, JS Variables

5, JS Data Types

< p>6, JS Functions

7, JS Operators

8, Flow Control

9, JS Errors and Debugging

JavaScript Objects and Scopes

1, Number Number

2, String String

3, Date Date

4, Array

5, Math

6, DOM Objects and Events

7, BOM Objects

8, Window Objects

9, Scopes and Chaining of Scopes

10, JSON

Javascript Library

1, Jquery

2, Prototype

3, Ext Js

Jquery

1, Jquery basic syntax

2, Jquery selector

3, Jquery events

4, Jquery selector

5, Jquery effects and animations

6, using Jquery to manipulate HTML and DOM

7, Jquery traversal

8, Jquery wrapper functions

9, Jquery cases

Form Validation and Jquery Validate

1. Js validation of HTML forms

2, Jquery Validata basic usage

3, the default validation rules and tips

4, debug and ignore

5, change the error message display location and style

6, all the validation through the implementation of the function

7, modify the validation trigger

8, asynchronous validation

9, custom check methods

10, radio and checkbox, select validation

Java Basics

1, about Java

2, Java runtime mechanism

3, The first Java program, comments

4, Javac, Java, Javadoc and other commands

5, identifiers and keywords

6, the declaration of variables, initialization and application

7, the scope of the variable

8, the variable renamed

9, the basic data types

10, type conversion and type lifting

11, details of the use of various data types

12, escape sequences

13, the use of various operators

Process control

1, selective control statements if-else

2, selective control statements switch-case

3、Loop control statement while

4、Loop control statement do-while

5、Loop control statement for and enhanced for

6、break, continue, return

7、Loop tags

8、Declaration and initialization of arrays

9 Array memory space allocation

10, stack and heap memory

11, two-dimensional (multi-dimensional) arrays

12, Arrays class related methods

13, main method command-line arguments

Object-oriented

1, the basic idea of object-oriented

2, class and object

3, Member Variables and Default Values

4, Method Declaration, Invocation

5, Parameter Passing and Memory Map

6, Concept of Method Overloading

7, Principle of Invocation and Advantages of Overloading

8, Constructor Declaration and Default Constructor

9, Constructor Overloading

10, Use of the this keyword

11, this call constructor principle

12, the way the instance variable is initialized

13, variable parameter methods

Access control

1, packages packages and libraries

2, access modifiers private/ protected/public/package access rights

3, Class access rights

4, Abstract classes and abstract methods

5, Interfaces and implementations

6, Decoupling

7, Multiple inheritance in Java

8, Extending interfaces through inheritance

Errors and Exception Handling< /p>

1. Concepts: Errors and Exceptions

2. Basic Exceptions

3. Catching Exceptions catch

4. Creating Custom Exceptions

5. Catching All Exceptions

6. Java Standard Exceptions

7. Cleanup Using Finally

8. Limitations

9, Constructors

10, Exception Matching

11, Exception Usage Guide

Database Fundamentals (MySQL)

Database Fundamentals (MySQL)

JDBC

1, Jdbc Basic Concepts

2, Connecting to Databases with Jdbc

3, crud operations using Jdbc

4, multi-table operations using Jdbc

5, Jdbc driver types

6, Jdbc exceptions and batch processing

7, Jdbc stored procedures

Servlet and JSP

1, Servlet Introduction

2, Request object

3, Response object

4, forwarding and redirection

5, using Servlet to complete Crud

6, Introduction to Session and Coolie

7, ServletContext and Jsp

8, the use of El and Jstl

Ajax

1, what is Ajax

2, XMLHttpRequest object (XHR)

3, XHR request

4, XHR response

5. , readystate/onreadystatechange

6, Jquery Ajax

7, JSON

8, Case: server-side verification of username availability

Comprehensive case

1, the general process of project development introduction

2, Modularity and layering

3, DButils

4, QueryRunner

5, ResultSetHandle

6, Case: user login/registration, from the front-end to the back-end

Phase 2 Java SE

Access rights and inheritance

1, package declaration and use

2, import and import static

3, access rights modifiers

4, class encapsulation

5, static (static member variables)

6, final (modifier variables, methods)

7, static member variables Initialization methods

8, class inheritance and member inheritance

9, the use of super

10, call the parent class constructor

11, method overriding and variable hiding

12, inheritance to achieve polymorphism and type conversion

13, instanceof

Abstract class vs. Interfaces

1, Abstract Classes

2, Abstract Methods

3, Inheriting Abstract Classes

4, Abstract Classes and Polymorphism

5, Membership of Interfaces

6, Static Methods and Default Methods

7, Static Member Classes

8, Instance Member Classes

9, Local Classes

10, anonymous classes

11, the use of eclipse and debugging

12, the relationship between the inner class access to peripheral classes

13, the naming of the inner class

Lambda expressions and common classes

1, functional interfaces

2, the concept of Lambda expressions

3, Lambda expression applications

4, use cases

5, method references

6, enumerated types (compiler handling)

7, wrapped types (automatic unpacking and sealing)

8, String methods

9, Constant Pooling mechanism

10, String explained

11, StringBuilder explained

12, Math, Date use

13, Calendars use

Exception Handling and Generalization

1, Exception Classification

2, try-catch-finally

3, try-with-resources

4, multi-catchmulti-catch

5, throw vs. throws

6, custom exceptions and advantages

7, generalization background and advantages

8, parameterized types vs. native types

9

9, type inference

10, the difference between parameterized types and arrays

11, type wildcards

12, custom generic classes and type erasure

13, generic method overloading and rewriting

Collection

1 , commonly used data structures

2, Collection interface

3, List and Set interfaces

4, SortedSet and NavigableSet

5, the implementation of the relevant interface class

6, Comparable and Comparator

7, Queue interface

8, Deque interface

< p>9, Map interface

10, NavigableMap

11, the implementation of the relevant interfaces

12, streaming operations (aggregation operations)

13, the use of Collections class

I/O streams and reflections

1, the use of File class

2, byte streams

3, character streams

4, cache streams

5, conversion streams

6, data streams

7, object streams

8, class loading, linking and initialization

9, the use of ClassLoader

10, the use of Class class

11, call constructors through reflection

12, security manager

Network programming model and multi-threading

1, processes and threads

2, the way to create threads

3, thread-related methods

4, thread synchronization

5, thread deadlock

6, Thread Collaboration

7, Computer Networking (IP and Ports)

8, TCP and UDP Protocols

9, Methods Related to URLs

10, Accessing Network Resources

11, TCP Protocol Communication

12, UDP Protocol Communication

13, Broadcasting

SSM-Spring

1.Spring/Spring MVC

2.Creating a Spring MVC project

3.Spring MVC execution process and parameters

SSM-Spring.IOC

1. MVC

2. Creating a Spring MVC project

3. Spring MVC execution process and parameters

SSM-Spring.AOP

1. Spring MVC execution process and parameters

SSM-Spring.Mybatis

1.Introduction to MyBatis

2.MyBatis configuration file

3.CRUD with MyBatis

4.ResultMap use

5.MyBatis Associative Queries

6.Dynamic SQL

7.MyBatis Buffering

8.MyBatis-Generator

Socket Programming

1.Network Communications and Protocols

2.About Sockets

3. Java Socket

4. Socket Types

5. Socket Functions

6. WebSocket

7. WebSocket/Spring MVC/WebSocket Ajax

IO/Asynchronous

window Object

Global Scope

Window Relationships and Frames

Window Position and Size

Opening a Window

Interval and Timeout Calls (Flexible)

System Dialogs

location object

navigator object

screen object

history object

NIO/AIO

1.Network Programming Model

2.BIO/NIO/AIO

3.Synchronous Blocking

4.Synchronous Non-Blocking

5.Asynchronous Blocking

6.

7. NIO and AIO basic operations

8. High-performance IO design patterns

Third stage of the mainstream framework of Java

MyBatis

1. mybatis framework principles of analysis

2. mybatis framework for introductory programming

3. mybatis Mybatis and hibernate essential differences and application scenarios

4. mybatis development dao method

5. SqlMapConfig configuration file to explain

6. Input mapping-pojo wrapper type definition and implementation

7. Output mapping-resultType, resultMap

8. dynamic sql

9. order commodity data model analysis

10. advanced mapping

11. query cache of the first level of cache, second level of cache

12. mybatis and spring integration

13. mybatis reverse engineering Automatic code generation

Spring/Spring MVC

1. springmvc architecture introduction

2. springmvc introductory program

3. spring and mybatis integration

4. springmvc annotation development -Springmvc annotation development - RequestMapping annotations

6. springmvc annotation development - Controller method return value

7. . springmvc annotation development - springmvc parameter binding process analysis

8. springmvc annotation development - springmvc parameter binding example to explain

9. springmvc and struts2 difference

< p>10. springmvc exception handling

11. springmvc upload images

12. springmvc json interaction

13. springmvc RESTful support

14. springmvc interceptor

Phase 4 Relational Databases/MySQL/NoSQL

SQL Fundamentals

1. SQL and Mainstream Products

2. Download and Installation of MySQL (sinux/windows)

3. Basic Configuration/Configuration Files for MySql

4. Basic SQL Operations DDL

5. Basic SQL Operations DML

6. Basic SQL Operations DCL

7. MySQL Client Tools

8. MySQL Help Documentation

MySQL Data Types and Operators

1 Numeric Types

2 Date-Time Types

3 String

12 Operator Prioritization

MySQL Functions

1 String Functions

2 Numeric Functions

3 Date and Time Functions

4 Process Functions

5 Other Common Functions

MySQL Storage Engines

1. MyISAM

3. InnoDB

4. Selecting the Right Storage Engine

Selecting the Right Data Type

1 CHAR and VARCHAR

2 TEXT and BLOB

3 Floating and Fixed Point Numbers

4 Date Type Selection

Character Sets

1 Overview of Character Sets

2 Unicode Character Sets

3 Chinese Characters and Some Common Character Sets

4 Choosing a Suitable Character Set

5 Character Sets Supported by MySQL

6 Setting Up MySQL Character Sets .

Design and Use of Indexes

1. What is an Index

2. Types of Indexes

3. Data Structure of Indexes BTree B+Tree Hash

4. Storage of Indexes

5. MySQL Indexes

6. Viewing Index Usage

7. .Index Design Principles

Views/Procedures/Functions/Triggers

1. What is a View

2. View Operations

3. What is a Stored Procedure

4. Stored Procedure Operations

5. What is a Function

6. Function Related Operations

7. Triggers

8. p>

Transaction Control/Locks

1. What is a Transaction

2. Transaction Control

3. Distributed Transactions

4. Locks/Table Locks/Row Locks

5. InnoDB Row Lock Contesting

6. Row Locking Patterns and Methods of Adding Locks in InnoDB7

7 InnoDB Row Locks Implementation7

8 Gap Locks (Next-Key Locks)

9 Recovery and Replication Needs, Implications for InnoDB Locking Mechanisms

10 Differences in Consistent Reads and Locks at Different Isolation Levels in InnoDB

11 Table Locks

12 Deadbolts

SQL Mode and Security Questions

1. About SQL Mode

2. SQL Mode in MySQL

3. SQL Mode and Migration

4. SQL Injection

5. How to Avoid SQL Injection During Development

SQL Optimization

1. status command to understand the frequency of execution of various SQL

2. locate the execution of less efficient SQL statements

3. EXPLAIN to analyze the execution plan of inefficient SQL

4. identify the problem and take appropriate optimization measures

5. indexing problems

6. regularly analyze the table and check the table

7. Optimizing Tables Regularly

8. Optimizing Commonly Used SQL

Optimizing MySQL Database Objects

1. Optimizing Table Data Types

2 Hashing

3 Inverse Normalization

4 Using Intermediate Tables to Improve Statistical Querying

5. Important Parameters that Affect MySQL Performance Parameters

6. Impact of Disk I/O on MySQL Performance

7. Using Connection Pooling

8. Reducing the Number of MySQL Connections

9. MySQL Load Balancing

MySQL Clustering

MySQL Administration and Maintenance

MemCache

Redis

Redis, MySQL, and MySQL. p>Redis

Using MemCache and Redis in Java Projects

Phase 5: Operating Systems/Linux, Cloud Architecture

Linux Installation and Configuration

1. Installing Linux to a Hard Drive

2. Getting Information and Searching Applications

3, Advanced: Repairing a Damaged Grub

4. About Superuser root

5. Distribution-Dependent System Management Tools

6. About Hardware Drivers

7. Advanced: Configuring Grub

System Administration and Directory Management

1. Basic Shell Commands

2.

3, find command, locate command

4, find a specific program: whereis

5, the architecture of the Linux file system

6, move, copy, and delete

7, file and directory permissions

8, file type vs. Input and output

9, vmware introduction and installation and use

10, network management, partition mount

User and user group management

1, package management

2, disk management

3, advanced hard disk management RAID and LVM

4, advanced: backup your work and System

5, Users and User Groups Basics

6, Managing, Viewing, and Switching Users

7, /etc/... File

8, process management

9, linux VI editor, awk, cut, grep, sed, find, unique, etc.

Shell Programming

1, SHELL variables

2, pass parameters

3, arrays and operators

4 SHELL commands

5, SHELL process control

6, SHELL functions

7, SHELL input/output redirection

8, SHELL file contains

Server Configuration

1, the system boot

2, managing daemons

< p>3, start SSH service via xinetd

4, configure inetd

5, Tomcat installation and configuration

6, MySql installation and configuration

7, deploying the project to Linux

Phase 6: Hadoop ecosystem

Hadoop Foundation

1, Introduction to Big Data

2, Google and Hadoop modules

3, Hadoop ecosystem

4, Introduction to Hadoop common projects

5, Hadoop environment installation and configuration

6, Hadoop installation mode

7, Hadoop configuration files

HDFS Distributed File System

1. Recognizing HDFS and its HDFS architecture

2. Hadoop's RPC mechanism

3. HDFS's HA mechanism

4. HDFS's Federation mechanism

5, Hadoop file system access

6, JavaAPI interface and maintenance of HDFS

7, HDFS rights management

8, hadoop pseudo-distributed

Hadoop file I/O in detail

1, the data structure of the Hadoop file

2, HDFS Data Integrity

3, File Serialization

4, Hadoop's Writable Type

5, Hadoop Supported Compression Formats

6, Encoders and Decoders in Hadoop

7, Comparison of gzip, LZO and Snappy

8, HDFS Using shell + Java API

MapReduce works

1, MapReduce functional programming concepts

2, MapReduce framework structure

3, MapReduce operation principle

4, Shuffle stage and Sort stage < /p>

5. Task execution and job scheduler

6. Custom Hadoop scheduler

7. Asynchronous programming model

8. YARN architecture and its workflow

MapReduce programming

1. WordCount case study

2. Input format vs. Output Format

3, Compression Format and MapReduce Optimization

4, Auxiliary Classes and Streaming Interfaces

5, Secondary Sorting in MapReduce

6, Join Algorithm in MapReduce

7, Reading and Writing Data from MySQL

8, Hadoop system tuning

Hive data warehouse tools

1, Hive working principle, types and features

2, Hive architecture and its file format

3, Hive operations and Hive composite types

4, Hive's JOIN detailed

5, Hive optimization Strategy

6, Hive built-in operators and functions

7, Hive user-defined function interface

8, Hive's permission control

Hive in-depth explanation

1 , installation and deployment of Sqoop

2, Sqoop data migration

3, Sqoop use Case

4, in-depth understanding of the database import

5, export and transaction

6, export and SequenceFile

7, Azkaban execution workflow

Sqoop and Oozie

1 , install and deploy Sqoop

2, Sqoop Data Migration

3, Sqoop Use Cases

4, Deeper Understanding of Database Import

5, Export and Transactions

6, Export and SequenceFile

7, Azkaban Execution Workflow

Zookeeper in Detail

1, Introduction to Zookeeper

2. Download and deployment of Zookeeper

3. Configuration and operation of Zookeeper

4. Example of Zookeeper's local mode

5. Zookeeper's data model

6. Zookeeper command line operation Example

7, storm in Zookeeper directory structure

NoSQL, HBase

1, HBase features

2, HBase access interface

3, HBase storage structure and format

4, HBase design

5, Key algorithms and processes

6, HBase installation

7, HBase SHELL operation

8, HBase cluster build

Phase VII: Spark ecosystem

Spark

1. What is Spark

2. Spark big Data Processing Framework

3.Spark Features and Application Scenarios

4.Spark SQL Principles and Practices

5.Spark Streaming Principles and Practices

6.Getting Started with GraphX SparkR

7.Monitoring and Tuning for Spark

Spark Deployment and Operation

1.WordCount Preparation Development Environment

2.MapReduce Programming Interface Architecture

3.MapReduce Communication Protocols

4.Importing Hadoop's JAR File

5.MapReduce Code Implementation

6. p>

6. Packaging, deployment and running

7. Packaging into JAR files

Spark program development

1. Start the Spark Shell

2. Load the text file

3. RDD operations and their applications

4. RDD caching

5. Build the Eclipse development environment

6, build IntelliJ IDEA development environment

7, create SparkContext object

8, write compile and submit the application

Spark Programming Model

1, RDD features and dependencies

2, collection (array ) to create RDDs

3, storage to create RDDs

4, RDD transformations Execution Control operations

5, broadcast variables

6, accumulators

Job execution parse

1, Spark components

2, RDD views and DAG charts

3, based on the Standalone mode of Spark architecture

4, based on the YARN mode of Spark architecture

5, job event flow and scheduling analysis

6, build the application runtime environment

7, the application is converted to a DAG

Spark SQL and DataFrame

1. p>

1, Spark SQL architecture features

2, the difference between DataFrame and RDD

3, create operations DataFrame

4, RDD conversion to DataFrame

5, load and save operations and Hive table

6, Parquet file JSON dataset

7, distributed SQL Engine

8, performance tuning Data Types

Deeper Spark Streaming

1, Spark Streaming works

2, DStream programming model

3, Input DStream

4, DStream Transformation State Output

5, Optimizing Runtime and Memory Usage

6, File Input Sources

7, Receiver-Based Input Sources

8, Output Manipulation

Spark MLlib and Machine Learning

1, Machine Learning Classification Level Algorithms

2. Spark MLlib Library

3. MLlib Data Types

4. Algorithmic Libraries and Examples for MLlib

5. MLlib Key Concepts

6. Algorithmic Libraries and Examples

GraphX and SparkR

1, Spark GraphX architecture

2, GraphX programming and common graph algorithms

3, GraphX application scenarios

4, SparkR's working principle

5, R language communication with other languages

6, SparkR's operation and application

7, R's DataFrame operation method

8, SparkR's DataFrame

Scala programming and development

1, Scala syntax basics

2, idea tool installation

3, maven tool configuration

4, conditional structure, loop, advanced for loops

5, arrays, maps, tuples

6, classes, sample classes, objects, companion objects

7, higher-order functions and functional programming

Scala Advanced

1, Collateralization, closures

2, Pattern matching, bijective functions

3, type parameters

4, covariance and inversion

5, implicit conversion, implicit parameters, implicit values

6, Actor mechanism

7, advanced project examples

Python Programming

1, Introduction to Python Programming

2, Python's Basic Syntax

3, Python Development Environment Build

4, Pyhton development of Spark applications

Phase VIII: Storm ecosystem

storm introduction and basic knowledge

1, the birth of the birth and growth of storm

2, the advantages of the storm and the application of the storm

3, the basic knowledge of the concept of storm and configuration

4, serialization and fault-tolerant mechanisms

5, reliability mechanisms - guaranteed message processing

6, storm development environment and production environment

7, storm topology of the parallelism

8, storm command line client

Storm topology and components in detail

1, stream grouping and topology operation

2, the common patterns of topology

3, the local mode and stormsub comparison

4, using a non-jvm language to operate storm

5, hook, component basic interfaces

6, the basic abstract classes

7, Transaction interface

8, the interrelationship between components

spout details and bolt details

1, spout to get data

2, commonly used spout

3, learning to write spout class

4, bolt overview

5, reliable and unreliable bolt

5, reliable and unreliable bolt

6, basic abstract classes

7. Reliable bolt

6, composite streams and composite anchoring

7, using other languages to define bolt

8, learning to write bolt class

storm installation and cluster construction

1, storm cluster installation steps and preparation

2, local mode storm Configuration commands

3, configure the hosts file, install jdk

4, zookeeper cluster setup

5, deploy the node

6, storm cluster setup

7, zookeeper application cases

8, Hadoop highly available cluster setup

Kafka

1, Kafka introduction and installation

2, integration of Flume

3, Kafka API

4, the underlying principle of implementation of Kafka

5, Kafka's message processing mechanism

6, the definition of the transaction of the data transfer

7, Kafka's storage strategy

Flume

1, Flume introduction and installation

2, Flume Source explains

3, Flume Channel explains

4, Flume Sink explains

5, flume deployment types, stream configuration

6, a single layer of data transfer. p>

6, Single Agent, Multi-agent Explained

7, flume selector related configuration

Redis

1, Redis Introduction and Installation, Configuration

2, Redis Data Types

3, Redis Keys, Strings, Hashes

4, Redis lists and collections

5, Redis transactions and scripts

6, Redis data backup and recovery

7, Redis SHELL operations