🎞️ AV1_SymbolsAnalysis

Status: finished

The goal:

To analyze the AV1 software enconder results, from the replacement of the file Aom/aom_dsp/entenc.c with my colleague’s file @tuliopereirab available in his repository: arithmetic-encoder-av1. Thus generating when running AV1 an output file main_data.csv later used in Count_Symbols.py.

Count_Symbols.py:

This is the main python file, where it formats and organizes the values of the 5th and 6th columns of main_data.csv (respectively n and nsyms) in a matrix and creates from this a new matrix with the recurrence (%) of each symbol in each alphabet (or each s in each nsyms, as described in entenc.c). Description of how to run (also present as comment in the code itself):

• to run this code is necessary write 3 parameters in terminal
• first input cq20.csv file and second input cq55.csv file (different CQs files will be compared in the compilation output), like in under example:
  python Count_Symbols.py boat_cq20.csv boat_cq55.csv

Output:

boat.xlsx

	A	B	C	D	E	F	G	H	I	J	K	L	M	N	O	P	Q	R	S	T	U	V	W	X	Y	Z	AA	AB	AC	AD	AE	AF	AG	AH	AI	AJ
1	CQ 20	0	1	2	3	4	5	6	7	8	9	10	11	12	13	14	15			CQ 55	0	1	2	3	4	5	6	7	8	9	10	11	12	13	14	15
2	0	87041	85606	0	0	0	0	0	0	0	0	0	0	0	0	0	0			0	6375	6182	0	0	0	0	0	0	0	0	0	0	0	0	0	0
3	1	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0			1	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0
4	2	105345	78443	0	0	0	0	0	0	0	0	0	0	0	0	0	0			2	15989	19032	0	0	0	0	0	0	0	0	0	0	0	0	0	0
5	3	38934	7014	758	0	0	0	0	0	0	0	0	0	0	0	0	0			3	5612	1435	469	0	0	0	0	0	0	0	0	0	0	0	0	0
6	4	162191	83063	30835	47356	0	0	0	0	0	0	0	0	0	0	0	0			4	23471	6157	1762	1796	0	0	0	0	0	0	0	0	0	0	0	0
7	5	1693	724	2147	2032	2549	0	0	0	0	0	0	0	0	0	0	0			5	98	193	244	327	248	0	0	0	0	0	0	0	0	0	0	0
8	6	1625	301	627	1401	1579	1014	0	0	0	0	0	0	0	0	0	0			6	52	10	23	20	30	4	0	0	0	0	0	0	0	0	0	0
9	7	2045	3255	1977	5212	9803	5582	5271	0	0	0	0	0	0	0	0	0			7	482	517	358	1787	973	576	531	0	0	0	0	0	0	0	0	0
10	8	1036	230	492	519	868	942	552	260	0	0	0	0	0	0	0	0			8	561	101	109	270	134	70	28	21	0	0	0	0	0	0	0	0
11	9	236	37	425	136	388	360	273	57	21	0	0	0	0	0	0	0			9	328	32	149	79	172	74	28	1	0	0	0	0	0	0	0	0
12	10	2335	1457	334	2893	305	337	165	131	1542	210	0	0	0	0	0	0			10	1136	398	169	567	79	91	46	32	259	74	0	0	0	0	0	0
13	11	2583	695	463	307	138	96	69	11	9	1	0	0	0	0	0	0			11	284	122	95	86	73	55	27	19	22	13	0	0	0	0	0	0
14	12	2	2	53	64	74	31	40	15	49	50	32	35	0	0	0	0			12	0	0	12	1	3	3	1	0	8	1	4	4	0	0	0	0
15	13	4032	639	4032	196	303	228	868	827	185	1508	197	555	313	0	0	0			13	870	138	940	51	105	65	223	149	34	333	61	114	179	0	0	0
16	14	4604	326	2287	74	193	123	346	224	58	384	92	117	510	525	0	0			14	1552	108	774	6	8	3	6	5	0	4	1	3	151	16	0	0
17	15	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0			15	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0
18	16	639	172	188	48	387	44	224	248	207	168	171	105	312	148	216	283			16	21	14	3	8	6	7	8	19	18	7	17	7	8	5	2	7
19	Sum:	414341	261964	44618	60238	16587	8757	7808	1773	2071	2321	492	812	1135	673	216	283			Sum:	56831	34439	5107	4998	1831	948	898	246	341	432	83	128	338	21	2	7
20
21	% of symb	0	1	2	3	4	5	6	7	8	9	10	11	12	13	14	15			% of symb	0	1	2	3	4	5	6	7	8	9	10	11	12	13	14	15
22	0	50,42%	49,58%																	0	50,77%	49,23%
23	1																			1
24	2	57,32%	42,68%																	2	45,66%	54,34%
25	3	83,36%	15,02%	1,62%																3	74,67%	19,09%	6,24%
26	4	50,14%	25,68%	9,53%	14,64%															4	70,73%	18,55%	5,31%	5,41%
27	5	18,51%	7,92%	23,48%	22,22%	27,87%														5	8,83%	17,39%	21,98%	29,46%	22,34%
28	6	24,82%	4,60%	9,58%	21,40%	24,12%	15,49%													6	37,41%	7,19%	16,55%	14,39%	21,58%	2,88%
29	7	6,17%	9,82%	5,96%	15,72%	29,58%	16,84%	15,90%												7	9,23%	9,90%	6,85%	34,21%	18,63%	11,03%	10,16%
30	8	21,15%	4,69%	10,04%	10,59%	17,72%	19,23%	11,27%	5,31%											8	43,35%	7,81%	8,42%	20,87%	10,36%	5,41%	2,16%	1,62%
31	9	12,21%	1,91%	21,99%	7,04%	20,07%	18,62%	14,12%	2,95%	1,09%										9	38,01%	3,71%	17,27%	9,15%	19,93%	8,57%	3,24%	0,12%
32	10	24,05%	15,01%	3,44%	29,80%	3,14%	3,47%	1,70%	1,35%	15,88%	2,16%									10	39,85%	13,96%	5,93%	19,89%	2,77%	3,19%	1,61%	1,12%	9,08%	2,60%
33	11	59,08%	15,90%	10,59%	7,02%	3,16%	2,20%	1,58%	0,25%	0,21%	0,02%									11	35,68%	15,33%	11,93%	10,80%	9,17%	6,91%	3,39%	2,39%	2,76%	1,63%
34	12	0,45%	0,45%	11,86%	14,32%	16,55%	6,94%	8,95%	3,36%	10,96%	11,19%	7,16%	7,83%							12			32,43%	2,70%	8,11%	8,11%	2,70%		21,62%	2,70%	10,81%	10,81%
35	13	29,04%	4,60%	29,04%	1,41%	2,18%	1,64%	6,25%	5,96%	1,33%	10,86%	1,42%	4,00%	2,25%						13	26,67%	4,23%	28,82%	1,56%	3,22%	1,99%	6,84%	4,57%	1,04%	10,21%	1,87%	3,49%	5,49%
36	14	46,68%	3,31%	23,19%	0,75%	1,96%	1,25%	3,51%	2,27%	0,59%	3,89%	0,93%	1,19%	5,17%	5,32%					14	58,85%	4,10%	29,35%	0,23%	0,30%	0,11%	0,23%	0,19%		0,15%	0,04%	0,11%	5,73%	0,61%
37	15																			15
38	16	17,95%	4,83%	5,28%	1,35%	10,87%	1,24%	6,29%	6,97%	5,81%	4,72%	4,80%	2,95%	8,76%	4,16%	6,07%	7,95%			16	13,38%	8,92%	1,91%	5,10%	3,82%	4,46%	5,10%	12,10%	11,46%	4,46%	10,83%	4,46%	5,10%	3,18%	1,27%	4,46%

Combine_Sheets.py:

This is a secondary python file whose your only propouse is combine different .xlsx files in a only file divided by differents worksheet in the same file(however it loses formatting of the original file). Description of how to run (also present as comment in the code itself):

• To run this code you need at least 2 parameters in the terminal.
• first is the name of output xlsx file: output.xlsx file
• and the second argument is the file that will be attached to the output sheet example: boat.xlsx
• in that program one can write as many arguments as you please, and all the files added by argument will be added in the output sheet in differents worksheets.
  Example to run:
  python Combine_Sheets.py output.xlsx boat.xlsx dark.xlsx KristenAndSara.xlsx minecraft.xlsx Netflix_DrivingPOV.xlsx Netflix_RollerCoaster.xlsx Netflix_SquareAndTimelapse.xlsx Netflix_TunnelFlag.xlsx rush_hour.xlsx