Documentation


Viewing posts for the category Omarine User's Manual

Neural network: Using genetic algorithms to train and deploy neural networks: Embedding

Unlike normal data mining processes, embedded operation is in the interference area between data mining and machine learning. It has just exploited knowledge to put it into machine learning, and received training in data exploitation from the network. The mining is like a pipe where the flow of knowledge is progressing over time through training. That means the network has to learn knowledge and adjust the source of knowledge.

Embedded work has two effects:

Neural network: Using genetic algorithms to train and deploy neural networks: Selecting the class

I would like to say that the AI ​​era is just beginning, that there is plenty of room for you to be creative, especially young people. It is because of two reasons below:

Neural network: Using genetic algorithms to train and deploy neural networks: The probability distributions

Uniform distribution is fundamental, used in most cases such as to assign hybrid rates and mutation rates in genetic algorithms. The distribution can be applied directly like that, or as a basis for a subsequent process, like browsing a tree starting from the root. A specific application is to use in embedding techniques of neural networks.
Beside that, the Standard normal distribution, especially Truncated standard normal distribution, is useful in the initialization of neural networks.
We will in turn learn about them with full source code.

Uniform distribution

Uniform distribution is the most common, the probability to yield values ​​is equal within a range of their values. Fortunately, we already have many functions available to create this distribution, including erand48(). This function returns nonnegative double-precision floating-point values uniformly distributed over the interval [0.0, 1.0). Transforming to an arbitrary range is very simple, for example creating a uniform distribution within [-1.0, 1.0):

Neural network: Using genetic algorithms to train and deploy neural networks: The derivatives of the transfer functions

The two most common transfer functions (activation functions) are the sigmoid logistic function or sigmoid function and ReLU function. Their derivative formulas are as follows

1) sigmoid function

Neural network: Using genetic algorithms to train and deploy neural networks: Data mining with id3

We already know that although id3 is a legacy with limited capabilities, it is still the No. 1 candidate for distinguishing important features with a strong theoretical basis. So why not use it to remove irrelevant features?

REGULARIZATION

Regularization is a measure of transforming the problem into a basic form that can be solved by known methods, which can be applied when certain assumptions are satisfied. For example linearization of data in linear regression.
In machine learning (deep learning) there is no specific way to do that. It is desirable to remove less relevant or irrelevant features to simplify the problem. Since then name the method. A typical example is Google's "L1 Regularization" method, which has been detailed in the article Neural network: Using genetic algorithms to train and deploy neural networks: Need the test set? so I don't repeat it here. One thing can be seen immediately that the "L1 Regularization" method eliminates the less relevant features only after they have been put into the network. What do you think about this? Put noise into the network and then find a way to remove it! How many features are less relevant? Which ones?

If there are indeed less relevant features then id3 is a great way to remove them. This is done in the data mining step, ie before putting data into the network for training. You can identify these less relevant features by programming or using the tool fpp. Removed features will not be present in the id3's output