Skip to main content

Artificial Intelligence (support vector machine (SVM)) in OPENCV

A support vector machine (SVM) is a concept in computer science for a set of related supervised learning methods that analyze data and recognize patterns, used for classification and regression analysis. The standard SVM takes a set of input data and predicts, for each given input, which of two possible classes the input is a member of, which makes the SVM a non-probabilistic binary linear classifier. Given a set of training examples, each marked as belonging to one of two categories, an SVM training algorithm builds a model that assigns new examples into one category or the other. An SVM model is a representation of the examples as points in space, mapped so that the examples of the separate categories are divided by a clear gap that is as wide as possible. New examples are then mapped into that same space and predicted to belong to a category based on which side of the gap they fall on.



A Support Vector Machine (SVM) performs classification by constructing an N-dimensional hyperplane that optimally separates the data into two categories. SVM models are closely related to neural networks. In fact, a SVM model using a sigmoid kernel function is equivalent to a two-layer, perceptron neural network.  

 The accuracy of an SVM model is largely dependent on the selection of the model parameters.
 In the parlance of SVM literature, a predictor variable is called an attribute, and a transformed attribute that is used to define the hyperplane is called a feature. The task of choosing the most suitable representation is known as feature selection. A set of features that describes one case (i.e., a row of predictor values) is called a vector. So the goal of SVM modeling is to find the optimal hyperplane that separates clusters of vector in such a way that cases with one category of the target variable are on one side of the plane and cases with the other category are on the other size of the plane. The vectors near the hyperplane are the support vectors.

#include<opencv2/ml/ml.hpp>
class atsSVM
{
public:
    void svm(cv::Mat& trainingData, cv::Mat& trainingClasses, cv::Mat& testData, cv::Mat& testClasses) {


        //SVM Parameters
        CvSVMParams param = CvSVMParams();

        param.svm_type = CvSVM::C_SVC;
        param.kernel_type = CvSVM::RBF; //CvSVM::RBF, CvSVM::LINEAR ...
        param.degree = 0; // for poly
        param.gamma = 20; // for poly/rbf/sigmoid
        param.coef0 = 0; // for poly/sigmoid

        param.C = 7; // for CV_SVM_C_SVC, CV_SVM_EPS_SVR and CV_SVM_NU_SVR
        param.nu = 0.0; // for CV_SVM_NU_SVC, CV_SVM_ONE_CLASS, and CV_SVM_NU_SVR
        param.p = 0.0; // for CV_SVM_EPS_SVR

        param.class_weights = NULL; // for CV_SVM_C_SVC
        param.term_crit.type = CV_TERMCRIT_ITER +CV_TERMCRIT_EPS;
        param.term_crit.max_iter = 1000;
        param.term_crit.epsilon = 1e-6;

        // SVM training (use train auto for OpenCV>=2.0)
        CvSVM svm(trainingData, trainingClasses, cv::Mat(), cv::Mat(), param);

        cv::Mat predicted(testClasses.rows, 1, CV_32F);

        for(int i = 0; i < testData.rows; i++) {
            cv::Mat sample = testData.row(i);

            float x = sample.at<float>(0,0);
            float y = sample.at<float>(0,1);

            predicted.at<float>(i, 0) = svm.predict(sample);
        }
        float percentage = evaluate(predicted, testClasses) * 100;
        cout << "Support Vector Machine Evaluated Accuracy = " << percentage << "%" << endl;
    }
    void showplot(cv::Mat testData)
    {
        plot_binary(testData, prediction, "Predictions SVM");
    }
private:
        cv::Mat prediction;
};

 The major strengths of SVM are the training is relatively easy. No local optimal, unlike in neural networks. It scales relatively well to high dimensional data and the trade-off between classifier complexity and error can be controlled explicitly. The weakness includes the need for a good kernel function

Comments

  1. hi saharkiz

    thank you for your sharing. I want to ask a question. how I can calculate confidence of predicted value. I guess you used a function which name is evaluate.is it true ?

    thank you

    ReplyDelete
  2. hi saharkiz,
    thank you for this post. i am involved with English OCR app using opencv and c++. this post says about 2 possible classes, but i have 52 classes for both simple and capital letters. can you tell me how to change this code as a multi class SVM ?

    ReplyDelete

Post a Comment

Popular posts from this blog

Computing Entropy of an image (CORRECTED)

entropy is a measure of the uncertainty associated with a random variable. basically i want to get a single value representing the entropy of an image. 1. Assign 255 bins for the range of values between 0-255 2. separate the image into its 3 channels 3. compute histogram for each channel 4. normalize all 3 channels unifirmely 5. for each channel get the bin value (Hc) and use its absolute value (negative log is infinity) 6. compute Hc*log10(Hc) 7. add to entropy and continue with 5 until a single value converges 5. get the frequency of each channel - add all the values of the bin 6. for each bin get a probability - if bin 1 = 20 bin 2 = 30 then frequency is 50 and probability is 20/50 and 30/50 then compute using shannon formula  REFERENCE: http://people.revoledu.com/kardi/tutorial/DecisionTree/how-to-measure-impurity.htm class atsHistogram { public:     cv::Mat DrawHistogram(Mat src)     {         /// Separate the image in 3 places ( R, G and B )    

Artificial Intelligence (K Nearest Neighbor) in OPENCV

In pattern recognition , the k -nearest neighbor algorithm ( k -NN) is a method for classifying objects based on closest training examples in the feature space . k -NN is a type of instance-based learning , or lazy learning where the function is only approximated locally and all computation is deferred until classification. The k -nearest neighbor algorithm is amongst the simplest of all machine learning algorithms: an object is classified by a majority vote of its neighbors, with the object being assigned to the class most common amongst its k nearest neighbors ( k is a positive integer , typically small). If k = 1, then the object is simply assigned to the class of its nearest neighbor. The k -NN algorithm can also be adapted for use in estimating continuous variables. One such implementation uses an inverse distance weighted average of the k -nearest multivariate neighbors. This algorithm functions as follows: Compute Euclidean or Mahalanobis distance from target plo

Blob Detection, Connected Component (Pure Opencv)

Connected-component labeling (alternatively connected-component analysis, blob extraction, region labeling, blob discovery, or region extraction) is an algorithmic application of graph theory, where subsets of connected components are uniquely labeled based on a given heuristic. Connected-component labeling is not to be confused with segmentation. i got the initial code from this URL: http://nghiaho.com/?p=1102 However the code did not compile with my setup of OpenCV 2.2, im guessing it was an older version. so a refactored and corrected the errors to come up with this Class class atsBlobFinder     {     public:         atsBlobFinder()         {         }         ///Original Code by http://nghiaho.com/?p=1102         ///Changed and added commments. Removed Errors         ///works with VS2010 and OpenCV 2.2+         void FindBlobs(const cv::Mat &binary, vector < vector<cv::Point>  > &blobs)         {             blobs.clear();             // Fill the la