Metadata extraction with Apache Tika
At Hippo I work with/for customers that have quite a lot of content. The projects I work on have content in the range of 5.000 to 500.000 document gathered in one content repository. This can be just textual content, but most of the time this is a variety of different content types. You might think of images, PDFs and Microsoft office document formats. By default Apache JackRabbit , the layer underneath Hippo Repository , can indexes this kind of content by using extractors, so that the information can be found within the Hippo CMS 7 search or from any application connected to the Hippo Repository which is performing a search on the content repository. Being able to search on content found within a file is interesting, but there is so much more that you can do with this kind of information.
Having all this content inside the repository is nice, but a certain piece of content uploaded to the repository can contain much more information then the file itself and sometimes this metadata is ignored in content management systems. As an example you might want to be able to see the number of pages of a PDF document inside your CMS or view the EXIF information of an image stored inside your content repository. There are a number of parser libraries out there that can extract information from a specific file format, but you can get quite lost. Within the ASF there is also a very nice toolkit called Apache Tika , which provides parsers for a lot of different file formats.
What is Apache Tika?
Apache Tika is a subproject of the Apache Lucene project and is a toolkit for extracting content and metadata from different kind of file formats. The content extraction logic is not located inside Tika itself, but Tika defines a standard API and makes use of existing libraries like POI and PDFBox for it’s content extraction. While writing this post the current release of Tika is version 0.6 and the following file formats are already supported:
As you can see this is already quite a lot. The team behind Tika is working hard on improving the current parser possibilities and adding more formats for the upcoming releases. Tika is actually already being used by a number of other Apache projects like JackRabbit and Solr . Now let’s see how we can use Tika ourselves.
I always work with Maven as my build system, so let’s start of with a piece of pom.xml. First add the Tika parser dependency to our pom.xml.
By depending on tika-parsers Maven will automatically gather the required parser libraries, which are needed to parse certain file formats. Since my Java code example will be based on a unit test, we will also need to add JUnit as a dependency to our pom.xml.
In this post I want to see what kind of EXIF information can be retrieved from an image by using Tika. One of my hobbies is photography and therefor I have tons of images, which contain a lot of metadata about for instance the ISO speed or dimensions of an image. Now let’s write some actual code to see how Tika works.
As I mentioned before, my example code is written as a JUnit test. If you are not familiar with writing tests or JUnit itself please have a look at the JUnit website .
To be able to run this test, I’ve added one of my images on the test classpath, so my test class will be able to find the image resource. The following piece of code shows my entire test class.
public class ImageMetaDataTest { private static final String fileName = «IMG_2659.JPG»; private Tika tika; private InputStream stream; @Before public void setUp() { tika = new Tika(); stream = this.getClass().getResourceAsStream(fileName); } @Test public void testImageMetadataCameraModel() throws IOException, SAXException, TikaException { Metadata metadata = new Metadata(); ContentHandler handler = new DefaultHandler(); Parser parser = new JpegParser(); ParseContext context = new ParseContext(); String mimeType = tika.detect(stream); metadata.set(Metadata.CONTENT_TYPE, mimeType); parser.parse(stream,handler,metadata,context); assertTrue(»The expected Model is not correct», metadata.get(»Model»).equals(»Canon EOS 350D DIGITAL»)); } @After public void close() throws IOException { if(stream!=null) { stream.close(); } } }
As you can see the code is quite small. The most important part of the above code example is using the JpegParser to parse the .JPG file and the creation of the Metadata object with the appropriate information.
I think this simple test case shows you how easy to use the Tika API is. Of course in the above test case I only test for the current Camera Model, but the Metadata object holds much more information then just that.
Viewing all the fields found in the metadata of the image can be achieved quite easily by using for instance the following method.
private void listAvailableMetaDataFields(final Metadata metadata) { for(int i = 0; i
Source: