I want to classify certain data into different classes based on its contents. I did this using the naive bike classifier, and I get the output as the best category to which it belongs. But now I want to classify the news, other than those that were in the training set, into the class βothersβ. I cannot manually add all the data, other than the training data, to a specific class, because it has a huge number of other categories. Is there a way to classify other data?
private static File TRAINING_DIR = new File("4news-train");
private static File TESTING_DIR = new File("4news-test");
private static String[] CATEGORIES = { "c1", "c2", "c3", "others" };
private static int NGRAM_SIZE = 6;
public static void main(String[] args) throws ClassNotFoundException, IOException {
DynamicLMClassifier<NGramProcessLM> classifier = DynamicLMClassifier.createNGramProcess(CATEGORIES, NGRAM_SIZE);
for (int i = 0; i < CATEGORIES.length; ++i) {
File classDir = new File(TRAINING_DIR, CATEGORIES[i]);
if (!classDir.isDirectory()) {
String msg = "Could not find training directory=" + classDir + "\nTraining directory not found";
System.out.println(msg);
throw new IllegalArgumentException(msg);
}
String[] trainingFiles = classDir.list();
for (int j = 0; j < trainingFiles.length; ++j) {
File file = new File(classDir, trainingFiles[j]);
String text = Files.readFromFile(file, "ISO-8859-1");
System.out.println("Training on " + CATEGORIES[i] + "/" + trainingFiles[j]);
Classification classification = new Classification(CATEGORIES[i]);
Classified<CharSequence> classified = new Classified<CharSequence>(text, classification);
classifier.handle(classified);
}
}
}
source
share