Data-driven Video Codec with Implicit Neural Representations

# I. Introduction
Source: [https://arxiv.org/html/2607.15298](https://arxiv.org/html/2607.15298)
Data\-driven Video Codec with Implicit Neural Representations

Nishan Khanal\*, Saugat Neupane\*, Abhinav Chalise\*, Nimesh Gopal Pradhan\*, Dinesh Baniya Kshatri\*

Department of Electronics and Computer Engineering, Thapathali Campus

Institute of Engineering, Tribhuvan University, Kathmandu, Nepal

khanal\.nishan28@gmail\.com, saugatn3@gmail\.com, chalisezabhinav@gmail\.com,

nimeshgpradhan@gmail\.com, dinesh@ioe\.edu\.np

Abstract—A conventional codec stores a video as compressed pixel data\. We instead store the video, together with its audio track, as the weights of a single sinusoidal representation network \(SIREN\) that maps space–time coordinates to RGB values and audio amplitudes\. The network uses separate audio and video initialization layers, a stack of shared fully connected hidden layers, and three output branches: one for video and two Siamese audio branches whose disagreement is used to estimate and subtract residual noise\. The overfitted teacher network is then compressed by response\-based knowledge distillation into a smaller student, followed by 16\-bit symmetric weight quantization and lossless LZMA2 \(xz\) encoding\. On a 6\.08 MiB test video, the quantized student reaches a video PSNR of 28\.72 dB with SSIM of 0\.75, and an audio PSNR of 24\.18 dB with a log spectral distance of 10\.69 dB, while the pipeline shrinks the representation from 9\.05 MiB to 2\.33 MiB, an overall compression ratio of 2\.61\. A bit\-width sweep from 1\-bit to 32\-bit quantization shows that reconstruction quality saturates at 16 bits\. We compare against H\.264, HEVC, and MP3, report where the approach falls short of them, and describe a browser\-based prototype that trains, transfers, and decodes these models over WebRTC\.

Index Terms—Implicit neural representation, video compression, sinusoidal representation networks, Siamese SIREN, knowledge distillation, quantization\.

Video accounts for most of the data stored and moved on the modern internet, and the pressure on codecs keeps growing as 4K and 8K formats become ordinary\. The standards that carry this load, H\.264/AVC\[[1](https://arxiv.org/html/2607.15298#bib.bib1)\]and H\.265/HEVC\[[2](https://arxiv.org/html/2607.15298#bib.bib2)\], compress pixel data directly: they remove spatial and temporal redundancy with transforms, motion prediction, and entropy coding\. They are fast and mature, but they operate on a fixed representation of the signal — a grid of pixels — and their efficiency gains between generations have come at the cost of considerable encoder complexity\.

Implicit neural representations \(INRs\) take a different route\. Instead of storing samples of a signal, an INR trains a neural network to*be*the signal: the network maps a coordinate \(e\.g\., pixel position and frame time\) to the signal value at that coordinate, and the trained weights become the stored representation\. Sitzmann et al\. showed that multilayer perceptrons with sine activations \(SIRENs\) fit images, audio, and video far better than ReLU networks of the same size\[[3](https://arxiv.org/html/2607.15298#bib.bib3)\], and NeRV demonstrated that video compression can be recast as*model*compression once the video lives inside a network\[[4](https://arxiv.org/html/2607.15298#bib.bib4)\]\. Since then, INR\-based codecs have improved steadily\[[5](https://arxiv.org/html/2607.15298#bib.bib5),[6](https://arxiv.org/html/2607.15298#bib.bib6)\], and Siamese SIREN extended the idea to audio, using two lightly different network heads to estimate and remove reconstruction noise\[[7](https://arxiv.org/html/2607.15298#bib.bib7)\]\.

Almost all of this work treats video and audio separately\. A video INR stores the frames; the soundtrack is left to a conventional audio codec or to a second network\. NeRVA\[[8](https://arxiv.org/html/2607.15298#bib.bib8)\]is a notable exception, but it operates frame\-wise and mixes convolutional blocks with MLPs\. To our knowledge, no prior work represents both modalities*sample\-wise*in one plain MLP, where a single set of shared hidden layers serves both the pixel function and the amplitude function\.

This paper asks whether that unified representation is practical, and what it costs\. We make four contributions:

- •A unified audio–video SIREN\.One MLP takes normalized space–time coordinates\[\(x,y,t\),T\]\[\(x,y,t\),T\]and returns RGB values and audio amplitudes\. Audio and video enter through separate initialization layers \(they need very different first\-layer weight ranges\), share five hidden layers, and exit through one video branch and two Siamese audio branches used for noise estimation\.
- •A compression pipeline for the representation\.Response\-based knowledge distillation shrinks the 2\.37M\-parameter teacher to a 1\.30M\-parameter student; 16\-bit symmetric quantization halves the stored size again; LZMA2 encoding removes what redundancy remains\. The full pipeline takes the representation of one test video from 9\.05 MiB to 2\.33 MiB\.
- •A quantization study\.We quantize the student from 1 to 32 bits and measure PSNR, SSIM, and signal\-to\-quantization\-noise ratio\. Quality collapses below 10 bits and saturates at 16, which fixes our operating point\.
- •An end\-to\-end prototype\.A FastAPI backend trains and decodes models; browsers exchange the compressed weights peer\-to\-peer over WebRTC\.

We also report the negative results plainly\. The teacher network has a fixed size of about 9 MiB regardless of input, so short clips are*expanded*rather than compressed, and on most test content the codec trails H\.264 and HEVC at comparable quality\. The interesting property is that the stored size is constant in the video length and resolution — the codec gets relatively better as the input gets larger — and that a single cheap MLP carries both modalities at all\.

## II\. Related Work

Conventional codecs\.MPEG\-1 Layer 3 established perceptual audio coding, using psychoacoustic models and Huffman coding to discard inaudible detail\[[9](https://arxiv.org/html/2607.15298#bib.bib9)\]\. H\.264/AVC brought intra\-picture prediction, variable block sizes, multiple reference frames, and in\-loop deblocking to video\[[1](https://arxiv.org/html/2607.15298#bib.bib1)\]; HEVC roughly doubled its compression efficiency through more flexible block partitioning and better prediction\[[2](https://arxiv.org/html/2607.15298#bib.bib2)\]\. These standards are the baselines any learned codec must face\.

Learned video compression\.Li et al\. replaced residue coding with conditional coding, letting the encoder and decoder condition on learned feature\-domain context, and reported 26% bitrate savings over x265\[[10](https://arxiv.org/html/2607.15298#bib.bib10)\]; a later version with richer temporal and spatial contexts surpassed the ECM reference software\[[11](https://arxiv.org/html/2607.15298#bib.bib11)\]\. Liu et al\. compressed video in the latent space of a GAN\-trained autoencoder with a ConvLSTM predictor\[[12](https://arxiv.org/html/2607.15298#bib.bib12)\]\. Panneerselvam et al\. combined CNN\-based duplicate\-frame removal, GAN\-detected frame differences, and SVD to halve file sizes with about 10% quality loss\[[13](https://arxiv.org/html/2607.15298#bib.bib13)\]\.

Implicit neural representations\.SIREN showed that sine activations, with a principled initialization, let coordinate MLPs represent signals and their derivatives accurately\[[3](https://arxiv.org/html/2607.15298#bib.bib3)\]\. NeRV turned video coding into model fitting: a network maps a frame index to an RGB frame, and standard model\-compression tools then act as the video codec\[[4](https://arxiv.org/html/2607.15298#bib.bib4)\]\. Gomes et al\. added entropy\-constrained training so that rate and distortion are optimized jointly, removing the need for post\-hoc quantization\[[5](https://arxiv.org/html/2607.15298#bib.bib5)\], and Zhang et al\. boosted NeRV\-like representations with conditional decoders and consistent entropy minimization\[[6](https://arxiv.org/html/2607.15298#bib.bib6)\]\. On the audio side, Siamese SIREN found that two lightly perturbed SIREN heads make reconstruction noise estimable: the difference of the two outputs approximates the noise, which a spectral\-gating step then removes\[[7](https://arxiv.org/html/2607.15298#bib.bib7)\]\. NeRVA represents video and audio jointly, but frame\-wise and with convolutional blocks\[[8](https://arxiv.org/html/2607.15298#bib.bib8)\]\. KD\-INR applied knowledge distillation to INRs for time\-varying volumetric data, aggregating per\-timestep models into one compact network\[[14](https://arxiv.org/html/2607.15298#bib.bib14)\]\. Quantization tooling for all of these is well developed in mainstream frameworks\[[15](https://arxiv.org/html/2607.15298#bib.bib15)\]\.

Gap\.What is missing is a sample\-wise, MLP\-only representation in which one network jointly stores pixels and audio amplitudes and is then compressed as a unit\. That is the configuration this paper builds and measures\.

## III\. Method

### A\. SIREN Preliminaries

A SIREN layer applies a sine to an affine map,

xi\+1=sin⁡\(Wi​xi\+bi\),x\_\{i\+1\}=\\sin\(W\_\{i\}x\_\{i\}\+b\_\{i\}\),\(1\)whereWiW\_\{i\}andbib\_\{i\}are the weights and bias of layerii\. Because the sine and all of its derivatives are smooth and periodic, the network can fit high\-frequency content that ReLU MLPs smooth away, and training remains stable at depth\[[3](https://arxiv.org/html/2607.15298#bib.bib3)\]\. Weights are initialized uniformly in\[−c/n,c/n\]\[\-\\sqrt\{c/n\},\\sqrt\{c/n\}\]for fan\-innn, which keeps activations arcsine\-distributed from layer to layer, and the first layer is scaled by a frequency factorω0\\omega\_\{0\}so thatsin⁡\(ω0​W​x\+b\)\\sin\(\\omega\_\{0\}Wx\+b\)spans several periods over the normalized input range\. We use the standardω0=30\\omega\_\{0\}=30\.

### B\. Unified Audio–Video Architecture

![Refer to caption](https://arxiv.org/html/2607.15298v1/figs/siren_architecture.png)Figure 1:The unified SIREN\. Audio timestepTTand space–time coordinates\(x,y,t\)\(x,y,t\)enter through separate initialization layers, pass through shared sine\-activated hidden layers, and exit through one video branch and two Siamese audio branches\.Figure[1](https://arxiv.org/html/2607.15298#S3.F1)shows the network\. The input is a normalized coordinate tuple\[\(x,y,t\),T\]\[\(x,y,t\),T\]: pixel position\(x,y\)\(x,y\), frame indextt, and audio timestepTT, each scaled to\[−1,1\]\[\-1,1\]or\[0,1\]\[0,1\]as appropriate\. The output is the RGB value at\(x,y,t\)\(x,y,t\)and two audio amplitudes atTT\.

Two design points matter here\. First, audio and video need*separate initialization layers*\. Audio is far richer in high frequencies than video, and we found no single first\-layer weight range that serves both: the audio branch needs weights initialized in\[−25,25\]\[\-25,25\]to capture its spectrum, while the video branch needs the much narrower\[−2/3,2/3\]\[\-2/3,\\,2/3\]— wider ranges produced grainy frames and wrong colors, narrower ones muffled the audio\. So each modality gets its own first layer, and their outputs are concatenated before the shared stack\.

Second, the audio output is duplicated\. Following Siamese SIREN\[[7](https://arxiv.org/html/2607.15298#bib.bib7)\], two identical linear audio branches with different weight draws produce two estimatesf1f\_\{1\}andf2f\_\{2\}of the same amplitude\. Their difference,

noise^=f1−f2,\\widehat\{\\text\{noise\}\}=f\_\{1\}\-f\_\{2\},\(2\)estimates the reconstruction noise and drives the spectral denoising step of Section[III\-C\.3](https://arxiv.org/html/2607.15298#S3.SS3.SSS3)\. Figure[2](https://arxiv.org/html/2607.15298#S3.F2)sketches the idea\.

![Refer to caption](https://arxiv.org/html/2607.15298v1/figs/siamese_siren.png)Figure 2:Siamese SIREN principle: two branches infer the same audio; their difference estimates the noise\.The teacher uses five shared hidden layers of 512 neurons; the student, three\. In the student, all non\-hidden layers are reduced to 300 neurons while hidden layers stay at 512\. All layers use sine activations except the three output branches, which are linear\. Table[I](https://arxiv.org/html/2607.15298#S3.T1)gives the parameter breakdown: 2,369,029 trainable parameters for the teacher and 1,298,029 for the student\.

Table I:Trainable Parameters in the SIREN ModelsDimensionsTotalLayerTeacherStudentTeacherStudentAudio initial1×5121\\times 5121×3001\\times 3001,024600Video initial3×5123\\times 5123×3003\\times 3002,0481,200Shared \(first\)1024×5121024\\times 512600×512600\\times 512524,800307,712Shared \(rest\)4×\(512×512\)4\\times\(512\\times 512\)2×\(512×512\)2\\times\(512\\times 512\)1,050,624525,312Video branch512×512,512×3512\\times 512,\\,512\\times 3512×300,300×3512\\times 300,\\,300\\times 3264,195154,803Audio branch 1512×512,512×1512\\times 512,\\,512\\times 1512×300,300×1512\\times 300,\\,300\\times 1263,169154,201Audio branch 2512×512,512×1512\\times 512,\\,512\\times 1512×300,300×1512\\times 300,\\,300\\times 1263,169154,201Total2,369,0291,298,029
### C\. Codec Pipeline

![Refer to caption](https://arxiv.org/html/2607.15298v1/figs/system_block.png)Figure 3:System block diagram\. The transmitting end overfits the INR and compresses it by distillation, quantization, and encoding; the receiving end decodes, dequantizes, and runs inference to reconstruct the video with audio\.Figure[3](https://arxiv.org/html/2607.15298#S3.F3)shows the codec end to end\. Encoding a video means training a network on it; decoding means running inference over the coordinate grid\. The stages are as follows\.

#### 1\) Pre\-processing

The input container is split into video and audio streams \(Fig\.[4](https://arxiv.org/html/2607.15298#S3.F4)\)\. Frames are cropped and downsampled to a fixeda×ba\\times bresolution and their pixels normalized to\[0,1\]\[0,1\]; audio is resampled to a rate matched to the pixel count and normalized by its peak amplitude\. Because one frame contains far more pixels than there are audio samples in its display interval, audio samples are repeated until the two streams have equal length; the repetition is removed after decoding\. Pixel coordinates, frame indices, and audio timesteps are then normalized, producing training pairs of coordinates and target values\.

![Refer to caption](https://arxiv.org/html/2607.15298v1/figs/preprocessing.png)Figure 4:Pre\-processing pipeline\.
#### 2\) Overfitting and compression

The teacher is trained to overfit the coordinate\-to\-value mapping \(Section[III\-D](https://arxiv.org/html/2607.15298#S3.SS4)\)\. The compression pipeline then applies, in order: knowledge distillation into the student, 16\-bit symmetric quantization of weights and biases, and lossless LZMA2 encoding withxz\. The result is the transmitted bitstream\.

#### 3\) Decoding and post\-processing

The receiver decodes the LZMA2 stream, dequantizes weights using the stored per\-layer scale factors, and evaluates the network over the full coordinate grid\. RGB outputs are reassembled into frames \(Fig\.[5](https://arxiv.org/html/2607.15298#S3.F5)\)\. The two audio outputs are concatenated, their sample repetition removed, and their difference used as the noise estimate of \([2](https://arxiv.org/html/2607.15298#S3.E2)\)\. The denoiser computes the spectrogram statistics of the noise estimate \(per\-band mean and standard deviation\), forms a thresholdμ\+k​σ\\mu\+k\\sigmaper frequency band, masks the audio spectrogram below the threshold, smooths the mask with a Gaussian filter, and inverts the masked STFT to obtain the final audio\.

![Refer to caption](https://arxiv.org/html/2607.15298v1/figs/postprocessing.png)Figure 5:Post\-processing pipeline\.

### D\. Training Objective

The teacher minimizes a weighted sum of per\-modality reconstruction losses,

ℒcombined=λaudio​ℒaudio\+λvideo​ℒvideo,\\mathcal\{L\}\_\{\\text\{combined\}\}=\\lambda\_\{\\text\{audio\}\}\\,\\mathcal\{L\}\_\{\\text\{audio\}\}\+\\lambda\_\{\\text\{video\}\}\\,\\mathcal\{L\}\_\{\\text\{video\}\},\(3\)with

ℒaudio\\displaystyle\\mathcal\{L\}\_\{\\text\{audio\}\}=12​∑k=12∫Ωa‖Φa,k​\(T\)−fa​\(T\)‖​𝑑T,\\displaystyle=\\frac\{1\}\{2\}\\sum\_\{k=1\}^\{2\}\\int\_\{\\Omega\_\{a\}\}\\\!\\bigl\\\|\\Phi\_\{a,k\}\(T\)\-f\_\{a\}\(T\)\\bigr\\\|\\,dT,\(4\)ℒvideo\\displaystyle\\mathcal\{L\}\_\{\\text\{video\}\}=∫Ωv‖Φv​\(x,t\)−fv​\(x,t\)‖​𝑑x​𝑑t,\\displaystyle=\\int\_\{\\Omega\_\{v\}\}\\bigl\\\|\\Phi\_\{v\}\(x,t\)\-f\_\{v\}\(x,t\)\\bigr\\\|\\,dx\\,dt,\(5\)whereΦa,1,Φa,2\\Phi\_\{a,1\},\\Phi\_\{a,2\}are the two audio branch outputs,Φv\\Phi\_\{v\}the video output, andfaf\_\{a\},fvf\_\{v\}the ground\-truth amplitude and RGB functions\. In practice the integrals are MSE sums over the sampled coordinates\.

![Refer to caption](https://arxiv.org/html/2607.15298v1/figs/knowledge_distillation.png)Figure 6:Response\-based knowledge distillation\. The student is supervised by the ground truth \(hard loss\) and by the teacher’s outputs \(soft loss\)\.The student is trained by response\-based knowledge distillation \(Fig\.[6](https://arxiv.org/html/2607.15298#S3.F6)\): it sees both the ground truth and the frozen teacher’s outputs,

ℒKD=α​ℒhard\+\(1−α\)​ℒsoft,\\mathcal\{L\}\_\{\\text\{KD\}\}=\\alpha\\,\\mathcal\{L\}\_\{\\text\{hard\}\}\+\(1\-\\alpha\)\\,\\mathcal\{L\}\_\{\\text\{soft\}\},\(6\)where the hard loss compares student outputs with ground truth and the soft loss compares them with teacher outputs\. Both are split evenly between video and audio, and the audio half is split again between the two Siamese branches:

ℒ∙=0\.5​ℒvideo,∙\+0\.5​\(0\.5​ℒaudio,∙\+0\.5​ℒaudio\(siam\),∙\)\.\\mathcal\{L\}\_\{\\bullet\}=0\.5\\,\\mathcal\{L\}\_\{\\text\{video\},\\bullet\}\+0\.5\\bigl\(0\.5\\,\\mathcal\{L\}\_\{\\text\{audio\},\\bullet\}\+0\.5\\,\\mathcal\{L\}\_\{\\text\{audio\(siam\)\},\\bullet\}\\bigr\)\.\(7\)

### E\. Quantization and Encoding

Weights are quantized symmetrically to 16\-bit integers,

w′=round⁡\(ws\)\+z,w^\{\\prime\}=\\operatorname\{round\}\\\!\\left\(\\frac\{w\}\{s\}\\right\)\+z,\(8\)with zero\-pointz=0z=0and a per\-layer scalesschosen from the layer’s weight range; scale factors are stored with the model for dequantization\. Section[V\-E](https://arxiv.org/html/2607.15298#S5.SS5)justifies the choice of 16 bits empirically\. The quantized checkpoint is finally compressed with LZMA2 viaxz, which is lossless, so it changes no quality metric — only the file size\.

## IV\. Experimental Setup

### A\. Dataset

We evaluate on five short videos with audio, stored uncompressed in AVI/WAV so that no prior codec biases the ground truth\. Table[II](https://arxiv.org/html/2607.15298#S4.T2)lists their characteristics; all are448×256448\\times 256\. Videos 1 and 2 have temporally uncorrelated frames, chosen to stress the model when consecutive frames change completely; their audio is a violin melody \(1\.6 Hz–4\.4 kHz\) and a Nepali male speaker \(0\.3 Hz–5\.7 kHz\), respectively\. Videos 3–5 have correlated frames: a man dancing to music \(0\.1 Hz–15\.5 kHz\), people dancing to funk \(0\.4 Hz–10\.3 kHz\), and an English speech \(1\.2 Hz–10\.5 kHz\)\. Audio sample rates are set high so that fewer repetitions are needed to match the pixel count during pre\-processing\.

Table II:Video File CharacteristicsSNVideoFPSSample rate \(Hz\)Size \(MiB\)Duration \(s\)1Video 11114,6881\.9252Video 2196,0005\.33103Video 310114,6886\.0834Video 42114,68813\.6255Video 524114,68812\.43
### B\. Training Details

Everything is implemented in PyTorch and trained on a laptop with an Intel Core i9\-13980HX, 24 GiB of RAM, and an NVIDIA RTX 4090 laptop GPU with 16 GiB of VRAM\. Table[III](https://arxiv.org/html/2607.15298#S4.T3)lists the hyperparameters\. The teacher trains for 7,000 epochs at a learning rate of10−510^\{\-5\}; the student for 6,000 epochs, starting at10−410^\{\-4\}for the first 1,000 epochs and dropping to10−510^\{\-5\}for the rest\. Both use Adam andω0=30\\omega\_\{0\}=30, and the checkpoint with the lowest loss is kept\. Training continues well past the point where the loss flattens because the perceptual metrics \(PSNR, SSIM, LPIPS, LSD\) keep improving after the raw MSE has nearly converged\.

Table III:Hyperparameters for the Teacher and Student ModelsHyperparameterTeacherStudentEpochs7,0006,000ω0\\omega\_\{0\}30First\-layer weights, audio \(βa\\beta\_\{a\}\)\(−25,25\)\(\-25,\\,25\)First\-layer weights, video \(βv\\beta\_\{v\}\)\(−2/3,2/3\)\(\-2/3,\\,2/3\)Learning rate \(η\\eta\)1×10−51\\times 10^\{\-5\}10−410^\{\-4\}\(first 1,000 ep\.\),then10−510^\{\-5\}OptimizerAdam
### C\. Metrics

Video quality is measured with PSNR, SSIM, and LPIPS against the source frames, and with SQNR when comparing a quantized model with its unquantized counterpart\. Audio quality is measured with PSNR, log spectral distance \(LSD\), and ViSQOL, an objective estimate of perceived listening quality on a 1–5 scale\. File sizes and compression ratios \(source size divided by stored model size\) complete the picture\.

## V\. Results and Analysis

### A\. Training Behavior

![Refer to caption](https://arxiv.org/html/2607.15298v1/figs/teacher_loss_v1.png)\(a\)Teacher, Video 1\.
![Refer to caption](https://arxiv.org/html/2607.15298v1/figs/student_loss_v1.png)\(b\)Student, Video 1\.

Figure 7:Training loss curves\. Both models drop steeply within the first few hundred epochs and then improve slowly; perceptual metrics keep improving after the loss flattens\.Figure[7](https://arxiv.org/html/2607.15298#S5.F7)shows representative loss curves\. The teacher’s loss falls steeply and is near zero around epoch 500; the student stabilizes even earlier, near epoch 200\. The long tail of training is not wasted: PSNR, SSIM, LPIPS, and LSD continue to improve while the loss changes only in its low digits, which is why we train to 7,000 and 6,000 epochs rather than stopping at loss convergence\.

### B\. Teacher Model Quality

Table IV:Video Metrics of the Teacher ModelVideoPSNR \(dB\)LPIPSSSIMOrig\. \(KiB\)Model \(KiB\)RatioVideo 142\.690\.050\.981,9729,2600\.21Video 235\.080\.170\.925,4620\.58Video 321\.880\.390\.676,2350\.67Video 426\.890\.410\.7214,0181\.50Video 532\.880\.210\.9212,7811\.37Table V:Audio Metrics of the Teacher ModelVideoPSNR \(dB\)LSD \(dB\)ViSQOLVideo 157\.574\.503\.54Video 257\.094\.283\.35Video 362\.507\.604\.51Video 464\.338\.153\.58Video 546\.606\.892\.66Tables[IV](https://arxiv.org/html/2607.15298#S5.T4)and[V](https://arxiv.org/html/2607.15298#S5.T5)report teacher quality\. Video PSNR spans 21\.88 dB \(Video 3, fast dance motion\) to 42\.69 dB \(Video 1, static frames\), with SSIM between 0\.67 and 0\.98\. Audio reconstruction is strong across the board — 46\.6 to 64\.3 dB PSNR, with ViSQOL up to 4\.51 on Video 3\.

The model column exposes the codec’s defining property: the stored size is 9,260 KiB for*every*video, because the network, not the content, sets the size\. For Videos 1–3 that means a compression ratio below 1 — the “compressed” file is larger than the source\. For the two largest inputs the ratio climbs past 1\.3\. The size floor is a fixed cost that only pays off once the input is big enough, and everything in Section[V\-D](https://arxiv.org/html/2607.15298#S5.SS4)is about lowering that floor\.

### C\. Audio Fidelity

![Refer to caption](https://arxiv.org/html/2607.15298v1/figs/fft_spectrogram_v1.png)Figure 8:FFT and spectrogram comparison of ground\-truth and predicted audio for Video 1\.Figure[8](https://arxiv.org/html/2607.15298#S5.F8)compares the ground\-truth and reconstructed audio of Video 1 in the frequency and time–frequency domains\. The FFT envelopes match closely with small deviations, and the spectrograms are nearly identical; the same holds for the other four videos\. The one systematic artifact appears in Videos 2 and 5, where the spectral\-gating denoiser misclassifies quiet parts of the signal as noise and removes them — visible as small gaps in the spectrogram\. The noise estimate from the Siamese branches is what makes the denoiser work at all, but its threshold is a blunt instrument on speech pauses\.

### D\. Distillation and 16\-bit Quantization

Table VI:Student Model Before and After 16\-bit Quantization, Video 1MetricStudent16\-bit quantizedPSNR, frames \(dB\)35\.552535\.3520SSIM0\.91940\.9189LPIPS0\.07940\.0848PSNR, audio \(dB\)18\.792918\.8487LSD \(dB\)4\.59304\.8484SQNR \(dB\)—39\.214File size \(MiB\)4\.962\.48Compression ratio0\.380\.77Table VII:Student Model Before and After 16\-bit Quantization, Video 3MetricStudent16\-bit quantizedPSNR, frames \(dB\)28\.858828\.7153SSIM0\.75790\.7529LPIPS0\.40390\.4126PSNR, audio \(dB\)24\.199224\.1786LSD \(dB\)10\.703810\.6928File size \(MiB\)4\.962\.48Compression ratio1\.222\.45Tables[VI](https://arxiv.org/html/2607.15298#S5.T6)and[VII](https://arxiv.org/html/2607.15298#S5.T7)show the student before and after quantization on Videos 1 and 3\. Distillation cuts the checkpoint from 9\.05 MiB to 4\.96 MiB, and quantization halves it to 2\.48 MiB\. The quality cost is small: on Video 1 the frame PSNR moves from 35\.55 to 35\.35 dB and SSIM from 0\.9194 to 0\.9189; on Video 3 the changes are similarly marginal\. The SQNR of 39\.2 dB confirms that 16\-bit rounding injects little noise relative to the signal\.

Two effects deserve honest mention\. First, on Video 3 the student’s*video*quality \(28\.86 dB\) exceeds the teacher’s \(21\.88 dB\), but its*audio*quality drops sharply \(24\.2 dB against the teacher’s 62\.5 dB\)\. A plausible reading is that the capacity budget shifts toward video during distillation and the audio pays for it, though we have not isolated the cause\. Second, audio PSNR on Video 1*improves*slightly after quantization \(18\.79 to 18\.85 dB\) while LSD worsens \(4\.59 to 4\.85 dB\); the rounding acts as weak regularization on amplitude error while blurring the spectrum\. Neither effect changes the overall picture: the compression is nearly free at 16 bits\.

### E\. Quantization Bit\-width Study

![Refer to caption](https://arxiv.org/html/2607.15298v1/figs/filesize_quant.png)\(a\)File size\.
![Refer to caption](https://arxiv.org/html/2607.15298v1/figs/psnr_quant.png)\(b\)Average PSNR\.
![Refer to caption](https://arxiv.org/html/2607.15298v1/figs/ssim_quant.png)\(c\)Average SSIM\.
![Refer to caption](https://arxiv.org/html/2607.15298v1/figs/sqnr_quant.png)\(d\)SQNR\.

Figure 9:Effect of quantization bit\-width on the Video 1 student model\. Quality metrics stabilize from 16 bits onward\.![Refer to caption](https://arxiv.org/html/2607.15298v1/figs/frame_predictions.png)Figure 10:Frame 10 of Video 1 reconstructed by the student model quantized to 1, 4, 7, 10, 14, 16, 18, and 32 bits\.To pick the operating bit\-width we quantized the Video 1 student to every precision from int1 to int32 and measured average PSNR, SSIM, and SQNR over five reference frames \(Fig\.[9](https://arxiv.org/html/2607.15298#S5.F9)\); file sizes could be exported for int8, int16, float16, and int32\. Quality degrades severely at low bit\-widths and recovers as precision grows, but from 16 bits onward the curves are flat — more precision buys nothing visible\. Figure[10](https://arxiv.org/html/2607.15298#S5.F10)makes the same point with images: 1\-bit and 4\-bit outputs are unrecognizable, 7–14 bits progressively sharpen, and 16, 18, and 32 bits are indistinguishable to the eye\. We therefore fix 16\-bit quantization for the pipeline\.

![Refer to caption](https://arxiv.org/html/2607.15298v1/figs/hist_v1.png)Figure 11:Weight and bias histograms of the Video 1 student model, by layer group\.Figure[11](https://arxiv.org/html/2607.15298#S5.F11)shows the student’s weight and bias distributions by layer group\. Weights are roughly Gaussian around zero — friendly territory for symmetric quantization — while biases are more uniform or lightly skewed\.

### F\. Lossless Encoding

Table VIII:Effect of LZMA2 \(xz\) Encoding on the Quantized StudentVideo 1Video 3MetricBeforeAfterBeforeAfterFile size \(MiB\)2\.482\.432\.482\.33Compression ratio0\.770\.792\.452\.61LZMA2 encoding is lossless, so quality metrics are untouched; Table[VIII](https://arxiv.org/html/2607.15298#S5.T8)shows the size effect\. The gain is modest — 2\.48 to 2\.43 MiB for Video 1 and 2\.48 to 2\.33 MiB for Video 3 — because quantized network weights have little of the byte\-level redundancy that dictionary coders exploit\. Video 3 compresses slightly better, suggesting more redundancy in its weight stream\. The final figure for Video 3 is an end\-to\-end compression ratio of 2\.61 against the 6\.08 MiB source\.

### G\. Comparison with Conventional Codecs

Table IX:Video Metrics of Conventional Codecs, Video 1 \(1,972 KiB\)CodecCRFBitrate\(kbps\)PSNR\(dB\)SSIMLPIPSSize\(KiB\)RatioH\.264/MP3132061\.600\.990\.004429\.644\.592319247\.240\.990\.005178\.5911\.04516432\.000\.830\.2344\.2844\.52H\.265/MP3132059\.530\.990\.004378\.325\.212819245\.320\.980\.02154\.5112\.76516433\.370\.850\.2145\.7043\.14Table X:Video Metrics of Conventional Codecs, Video 3 \(6,235 KiB\)CodecCRFBitrate\(kbps\)PSNR\(dB\)SSIMLPIPSSize\(KiB\)RatioH\.264/MP3132058\.390\.980\.161,218\.825\.122319242\.220\.960\.16161\.8538\.52516422\.170\.720\.3431\.65196\.98H\.265/MP3132056\.070\.990\.16861\.807\.232819240\.200\.960\.18113\.0255\.16516424\.160\.730\.3531\.42198\.39Table XI:Audio Metrics of MP3, Videos 1 and 3Video 1Video 3Bitrate\(kbps\)PSNR\(dB\)LSDViSQOLPSNR\(dB\)LSDViSQOL6435\.341\.974\.6627\.994\.214\.6419243\.280\.424\.7342\.230\.754\.7332064\.780\.104\.7366\.450\.184\.73We encoded all five videos with H\.264 and H\.265 \(with MP3 audio\) at three CRF settings each; Tables[IX](https://arxiv.org/html/2607.15298#S5.T9)–[XI](https://arxiv.org/html/2607.15298#S5.T11)show Videos 1 and 3, and the remaining videos follow the same pattern\. The conventional codecs win on compression ratio by one to two orders of magnitude, which is the expected outcome against three decades of engineering\.

The quality comparison is more nuanced\. The teacher sits slightly below H\.264/HEVC at CRF 23–28 on Videos 1, 2, and 5, and is comparable to their CRF 51 output on Videos 3 and 4\. Its audio trails MP3 at 320 kbps on Videos 1–4 and matches MP3 at 192 kbps on Video 5\. The quantized student matches the CRF 51 tier on Video 1 and slightly beats it on Video 3 \(28\.72 dB against 22\.17–24\.16 dB\), while its audio lands near MP3 at 64 kbps\. The INR codec is thus competitive only at the aggressive end of the conventional codecs’ range — but it reaches that point with a stored size that does not depend on video length or resolution, whereas conventional bitstreams scale linearly with content\.

### H\. Prototype Deployment

The full pipeline runs behind a web interface\. A FastAPI backend accepts an uploaded video, trains the INR in a subprocess, and returns the compressed model; browsers exchange models directly over WebRTC data channels, with WebSocket signaling and a public STUN server for NAT traversal, so transfers inside a network need no central storage\. Received models are dequantized and inferenced server\-side, and the reconstructed video plays back in the browser next to the original for visual comparison\.

## VI\. Limitations and Future Work

The measurements point at four concrete weaknesses\. The fixed model\-size floor \(2\.33–2\.48 MiB after full compression\) makes the codec useless for short clips and only mildly useful at the sizes we tested; the floor must drop, whether by smaller architectures, pruning, or entropy\-constrained training of the kind proposed by Gomes et al\.\[[5](https://arxiv.org/html/2607.15298#bib.bib5)\]\. Dynamic content is the second weakness: reconstruction quality falls visibly as motion and frame rate increase, and representing high\-fps video well remains open\. Third, distillation currently trades audio quality for video quality; a loss schedule that protects the audio branch, or separate distillation temperatures per modality, is worth trying\. Fourth, encoding means training a network per video for thousands of epochs, which rules out real\-time use; meta\-learned initializations or shared backbones across videos could cut this cost\. On the quantization side, 16\-bit symmetric rounding is simple but leaves room: quantization\-aware training and mixed per\-layer precision could push below 16 bits without the quality cliff we observed\.

## VII\. Conclusion

We built and measured a codec that stores video and audio jointly as the weights of one sine\-activated MLP\. Separate per\-modality initialization layers turned out to be necessary — audio wants first\-layer weights in\(−25,25\)\(\-25,25\), video in\(−2/3,2/3\)\(\-2/3,2/3\)— and a Siamese audio head provides a usable noise estimate for spectral denoising at decode time\. Knowledge distillation, 16\-bit symmetric quantization, and LZMA2 encoding together cut the stored representation from 9\.05 MiB to 2\.33 MiB while leaving reconstruction quality essentially unchanged, and a sweep over bit\-widths shows 16 bits is the knee of the quality curve\. Against H\.264, HEVC, and MP3 the approach is competitive only at their lowest\-quality settings, and its fixed size floor penalizes short inputs\. What it offers in exchange is a single, differentiable, resolution\-independent representation of both modalities whose stored size is constant in the content length — a property no conventional codec has, and one that becomes more valuable as inputs grow\. Lowering the size floor and handling motion are the clear next steps\.

## References

- \[1\]H\. Kalva, “The H\.264 video coding standard,”*IEEE MultiMedia*, vol\. 13, no\. 4, pp\. 86–90, 2006\.
- \[2\]M\. T\. Pourazad, C\. Doutre, M\. Azimi, and P\. Nasiopoulos, “HEVC: The new gold standard for video compression: How does HEVC compare with H\.264/AVC?”*IEEE Consumer Electronics Magazine*, vol\. 1, no\. 3, pp\. 36–46, 2012\.
- \[3\]V\. Sitzmann, J\. N\. P\. Martel, A\. W\. Bergman, D\. B\. Lindell, and G\. Wetzstein, “Implicit neural representations with periodic activation functions,” in*Advances in Neural Information Processing Systems \(NeurIPS\)*, 2020\.
- \[4\]H\. Chen, B\. He, H\. Wang, Y\. Ren, S\.\-N\. Lim, and A\. Shrivastava, “NeRV: Neural representations for videos,” in*Advances in Neural Information Processing Systems \(NeurIPS\)*, 2021\.
- \[5\]C\. Gomes, R\. Azevedo, and C\. Schroers, “Video compression with entropy\-constrained neural representations,” in*Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition \(CVPR\)*, 2023, pp\. 18 497–18 506\.
- \[6\]X\. Zhang, R\. Yang, D\. He, X\. Ge, T\. Xu, Y\. Wang, H\. Qin, and J\. Zhang, “Boosting neural representations for videos with a conditional decoder,” 2024, arXiv:2402\.18152\.
- \[7\]L\. A\. Lanzendörfer and R\. Wattenhofer, “Siamese SIREN: Audio compression with implicit neural representations,” 2023, arXiv:2306\.12957\.
- \[8\]A\. Choudhury, P\. Singh, and G\.\-M\. Su, “NeRVA: Joint implicit neural representations for videos and audios,” in*2024 IEEE International Conference on Multimedia and Expo \(ICME\)*, 2024, pp\. 1–6\.
- \[9\]S\. Shlien, “Guide to MPEG\-1 audio standard,”*IEEE Transactions on Broadcasting*, vol\. 40, no\. 4, pp\. 206–218, 1994\.
- \[10\]J\. Li, B\. Li, and Y\. Lu, “Deep contextual video compression,” in*Advances in Neural Information Processing Systems \(NeurIPS\)*, 2021\.
- \[11\]J\. Li, B\. Li, and Y\. Lu, “Neural video compression with diverse contexts,” in*Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition \(CVPR\)*, 2023\.
- \[12\]B\. Liu, Y\. Chen, S\. Liu, and H\.\-S\. Kim, “Deep learning in latent space for video prediction and compression,” in*Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition \(CVPR\)*, 2021, pp\. 701–710\.
- \[13\]K\. Panneerselvam, K\. Mahesh, V\. Josephine, and R\. Anandan, “Effective and efficient video compression by the deep learning techniques,”*Computer Systems Science and Engineering*, vol\. 45, pp\. 1047–1061, 2022\.
- \[14\]J\. Han, H\. Zheng, and C\. Bi, “KD\-INR: Time\-varying volumetric data compression via knowledge distillation\-based implicit neural representation,”*IEEE Transactions on Visualization and Computer Graphics*, vol\. 30, no\. 10, pp\. 6826–6838, 2024\.
- \[15\]PyTorch, “Quantization — PyTorch documentation,”[https://pytorch\.org/docs/stable/quantization\.html](https://pytorch.org/docs/stable/quantization.html), accessed 2024\-10\-15\.