Evaluating Compression Techniques for Computer Vision: An Open-Source Approach

The growing demand for computer vision applications relies heavily on image and video data, leading to an urgent need for effective compression techniques. As we continue to harness the power of visual data, finding methods to efficiently compress this information while preserving its integrity becomes increasingly critical. A team comprising Hyomin Choi from InterDigital, Heeji Han from Hanbat National University, Chris Rosewarne from Canon, and Fabien Racapé from InterDigital has introduced a promising solution: CompressAI-Vision. This open-source software platform is expertly designed to rigorously evaluate compression methods tailored for computer vision tasks.

CompressAI-Vision addresses the challenge of ensuring that video and image data remain highly effective for tasks like object detection, pose estimation, and tracking. Traditional video compression methods often fail to preserve the vital information required by artificial intelligence (AI) tasks, which has necessitated the creation of new evaluation approaches. This platform offers a standardized environment for testing various compression tools, focusing on their ability to maintain the accuracy of vision tasks, whether in local or remote processing contexts.

Machine Learning Focused Video Compression Evaluation

This innovative framework emphasizes the importance of machine learning performance by moving beyond classical metrics. CompressAI-Vision directly measures the effects of compression on the accuracy of AI models. By integrating with popular AI frameworks such as Detectron2 and MMPose, it provides a seamless way to evaluate performance post-compression and decompression.

The platform supports an extensive range of datasets frequently utilized in machine learning, including the OpenImages dataset, FLIR Thermal Datasets, SFU-HW-Objects, the Tencent Video Dataset (TVD), and Human in Events. This variety ensures that assessments are robust and relevant across different contexts and applications. Another significant feature of CompressAI-Vision is its compatibility with numerous modern video codecs like H.264, H.265, and H.266, allowing for comprehensive evaluation across different encoding methods.

Moreover, the framework is designed with an open-source model, inviting community input and customization. Researchers and developers can quickly adapt the platform for their specific datasets, AI models, and compression techniques. This flexibility empowers innovation in compression methodologies and paves the way for further enhancements in visual data handling.

CompressAI-Vision Evaluates Video Coding for Computer Vision

The scientific advancements showcased through CompressAI-Vision include extensive testing of standard codecs paired with various datasets. Research findings demonstrate significant compression efficiencies, particularly illustrated through comparisons between codecs like FCTM v6.1 and VCM-RS v0.12. Notably, the FCTM codec achieved outstanding bitrate reductions—79.35% for Class C and 69.02% for Class D within the SFU-HW-Obj dataset—while maintaining equivalent task accuracy.

The evaluation results reveal that, on average, the FCTM codec reduced bitrate by -58.33%, -41.43%, and -72.70% under Random Access, Low Delay, and All-Intra configurations, respectively, in contrast to VCM-RS’s outputs. When analyzing VCM-RS with respect to the FCM CTTC, the findings indicated that both VCM-RS and FCTM outperformed other methodologies on the TVD dataset, achieving near-lossless accuracy at elevated bitrates.

A deeper examination indicated that using VTM-23.3 as the inner codec for FCTM v6.1 produced superior results compared to JM-19.1 or HM-18.0. These insights illustrate CompressAI-Vision’s effectiveness in consistently evaluating coding performance across diverse inner codec configurations and varying inference pipelines.

Compression Evaluation for Computer Vision Tasks

CompressAI-Vision marks a significant stride in how we evaluate video compression techniques with a focus on computer vision applications. The platform permits comparative analysis of various coding tools while ensuring that task accuracy remains intact during evaluations under both remote and split inference settings. It allows detailed examination of the relationship between bitrate and task accuracy across different datasets, providing invaluable insights into the trade-offs between compression efficiency and performance.

The open-source nature of CompressAI-Vision not only promotes scalability but also encourages broader contributions from the research community, fostering an environment where ongoing development and innovation can flourish. The authors note that while the current iteration of the platform centers around convolutional neural networks, there are plans to extend support for vision transformer architectures. This will create opportunities for exploring how compression noise impacts embedding spaces, as well as optimizing coding methods for multi-task networks and diverse machine vision applications.