How to train a model for Sentence Detection in openNLP

How to train a model for Sentence Detection in openNLP

How to train a model for Sentence Detection in openNLP – In this tutorial, we shall understand how to train a model from input training data for Sentence Detection in openNLP using Java.

Why to train a model for Sentence Detection

There would always be a requirement for sentence detection. There could be new structure of statements in your use case or may be sentence detection has to be done for a language different from English or something that is readily not available. These scenarios would call out to build a model of our own, from our own training data, for our own purpose.

Train a model for Sentence Detection

Now let us see how to train a model for Sentence Detection in openNLP. Follow the below steps:

  1. Create a text file and keep a sentence for each line in the text file.
  2. Create an InputStreamFactory from the input file using code snippet shown below.
  3. Set the machine learning hyper parameters like number of iterations and cutoff using the code snippet shown below.
  4. With the help of train() method in SentenceDetectorME, generate a model.

Let us generate a model file with the help of training, as shown in below example:

Sentence Detector Training Example in openNLP

The following example SentenceDetectorTrainingExample.java shows how to train a model for your own training data. If you would like to know how to setup java project to use openNLP, in eclipse, refer to setup of java project with openNLP libraries, in eclipse. The process should be same, even for a different IDE(adding the required jars to the build path should do the magic).

Download ? SentenceDetectorTrainingExample.java & trainingDataSentences.txt

Download ? SentenceDetectorTrainingExample.java & trainingDataSentences.txt

When SentenceDetectorTrainingExample.java is run, the output to console is :

The project structure, training input file location and model file generation location, etc., for the example is shown below:

How to train a model for Sentence Detection in openNLP - project structure - Tutorialkart

Java Project Structure in Eclipse

 

 

Conclusion :

In this openNLP tutorial, we have completed on how to train a model for Sentence Detection in openNLP.