Multimedia Computing Project: Interactive 3D Media Gallery [V0.96]

[Please read Notes below.]

Important dates

Deliverable 1: Specification with 4 pages + images in annex if needed: April 18. Send to nmc@fct.unl.pt. With this deliverable the team members should be identified and should not change after this date.
Deliverable 2: Source code and final report: June 9, by email or using a service such as Google Drive. The visualizations and discussions will take place after.

Send the code to nmc@fct.unl.pt in zip format. The file should have the student numbers separated with "-". Example: 11111-2222.zip. It should include all the files required to compile and run the program. A final report should also be included. If large video files are needed please use a service such as Dropbox or Google Drive.

The project and report requirements are presented next.

Create a (preferably) 3D space where a user can move and interact with images and videos, displayed as virtual frames in a gallery, featuring overlays with extracted information such as luminance, edges and edge histogram information, dominant colors, while some algorithms like luminance, color, texture or keypoints enables automatic similarity detection, allowing users to navigate and group image categories based on visual features. Properties can also be revealed by touching an image or video.

Experiment with different behaviors for images or videos with different characteristics - images with dominant warm colors could move in one direction, images with many edges have a certain type of motion while videos with many cuts could have another.

Use simple based gesture recognition to navigate image/video galleries.

Images and videos should have the necessary metadata to support these operations, represented in XML format.

Please play attention to the data and results that you use in the project. Results should be coherent and aligned with the data that is used.

Deliverable 1: Specification

This is the initial specification of the system. The specification should include an interface sketch/storyboard as first approach to the interface, considering the requirements and features described above. User interface elements should be identified and described. The sketch can be done using several tools (e.g., pen based interfaces or even digitized drawings) and should be included in the document. This type of low fidelity prototypes provides an initial specification of the interface elements and the sequence of actions. This should also include a preliminary class structure.

Deliverable 2: Final Report + Code

The following structure is suggested for the final report (up to 6 pages), including part of the content from the initial specification:

Introduction
Related work
Algorithms and techniques (short description of the processing algorithms and visualization techniques)
Application implementation (short description of how it works)
Class description
Conclusions
References

The report should also include as appendix:

Class diagram, if not included in the report.
User manual (1-2 pages).

Notes [Updated 27/3/2025.]

28/3:

1 - At least one pixel processing algorithm (color or light) applied to the images and the result stored as metadata (Example: videoGrabberExample).

2 - Simple motion detection using the camera (Example: opencvExample).

3 - Use both methods and both addons to interface OpenCV.

21/3:

1 - A gallery with images and video (Examples: dirListExample, videoPlayerExample).

2 - The capability to play videos and show images full screen (Example: videoPlayerExample).

3 - The ability to show (and hide using a key) the camera capturing images (Example: videoGrabberExample).

4 - Face detection on the camera image (Example: opencvHaarfFinderExample).

5 - Read and write xml - one xml file for each image or video file with metadata (Example: xmlSettingsExample).

14/3: Introduction to openFrameworks and the project. Starting to experiment using the videoPlayer and the dirListExample. Both these examples are in the distribution in the /examples folder - in the video and input-output folders. Together, these two examples support some of the requirements in the project - namely displaying image and video files and listing a folder with this type of content. Both examples follow the standard OF model with setup(), update() and draw() methods. Also included event handlers, for example to handle key press or mouse move. The goal is to continue the example by displaying a gallery of images and videos. It should be possible to play, pause, resume the videos.

The goal for this class is to display images and video in the same window - like a gallery with rows and columns of image and video thumbnails.