spotify / luigiDuplication

Intro

For duplication, we look at places in code where there are 6 or more lines of code that are exactly the same.
Before duplication is calculated, the code is cleaned to remove empty lines, comments, and frequently duplicated constructs such as imports.
You should aim at having as little as possible (<5%) of duplicated code as high-level of duplication can lead to maintenance difficulties, poor factoring, and logical contradictions.

Learn more...

Duplication Overall

3% duplication:

18,669 cleaned lines of cleaned code (without empty lines, comments, and frequently duplicated constructs such as imports)
568 duplicated lines

48 duplicates

dependency graphs: 2D graph | 3D graph | 3D graph (with duplicates)...

Duplication per Extension

Duplication per Component (primary)

Longest Duplicates

The list of 48 longest duplicates.

See data for all 48 duplicates...

Size	#	Folders	Files	Lines	Code
18	x 2	luigi luigi	parameter.py parameter.py	1120:1137 (3%) 1271:1288 (3%)	view
18	x 2	luigi/contrib luigi/contrib	rdbms.py rdbms.py	168:190 (12%) 309:331 (12%)	view
17	x 2	luigi/contrib luigi/contrib	rdbms.py redshift.py	165:185 (12%) 628:648 (4%)	view
14	x 2	luigi/contrib luigi/contrib	rdbms.py redshift.py	309:326 (10%) 631:648 (3%)	view
12	x 2	luigi/contrib luigi/contrib	lsf_runner.py sge_runner.py	48:66 (40%) 60:78 (37%)	view
11	x 2	luigi/contrib luigi/contrib	mysqldb.py postgres.py	99:110 (8%) 235:246 (4%)	view
11	x 2	luigi/contrib luigi/contrib	mssqldb.py postgres.py	65:85 (17%) 197:218 (4%)	view
10	x 2	luigi/contrib luigi/contrib	postgres.py postgres.py	335:349 (4%) 463:477 (4%)	view
10	x 2	luigi/contrib/hdfs luigi/contrib/hdfs	format.py format.py	51:60 (8%) 82:93 (8%)	view
9	x 2	luigi/tools luigi/tools	range.py range.py	730:738 (2%) 787:795 (2%)	view
9	x 2	luigi/contrib luigi/contrib	redshift.py redshift.py	322:331 (2%) 346:355 (2%)	view
9	x 2	luigi/contrib luigi/contrib	redshift.py redshift.py	725:738 (2%) 799:812 (2%)	view
9	x 2	luigi luigi	format.py format.py	253:263 (2%) 421:431 (2%)	view
8	x 2	luigi/contrib luigi/contrib	gcs.py ssh.py	294:301 (2%) 208:215 (3%)	view
8	x 2	luigi/contrib luigi/contrib	mysqldb.py postgres.py	240:249 (6%) 418:429 (3%)	view
7	x 2	luigi/contrib luigi/contrib	mysqldb.py redshift.py	186:193 (5%) 732:738 (1%)	view
7	x 2	luigi/contrib/hdfs luigi/contrib/hdfs	format.py format.py	102:111 (6%) 120:129 (6%)	view
7	x 2	luigi/tools luigi/tools	range.py range.py	365:377 (1%) 435:447 (1%)	view
7	x 2	luigi/static/visualiser luigi/static/visualiser	index.html index.html	221:228 (1%) 322:329 (1%)	view
7	x 2	luigi/contrib luigi/contrib	redshift.py redshift.py	411:422 (1%) 799:810 (1%)	view
7	x 2	luigi/contrib luigi/contrib	mysqldb.py postgres.py	71:86 (5%) 203:219 (2%)	view
7	x 2	luigi/static/visualiser/css luigi/static/visualiser/css	luigi.css luigi.css	25:32 (3%) 35:42 (3%)	view
7	x 2	luigi/contrib luigi/contrib	mysqldb.py redshift.py	186:193 (5%) 806:812 (1%)	view
7	x 2	luigi/static/visualiser/js luigi/static/visualiser/js	visualiserApp.js visualiserApp.js	1342:1349 (<1%) 1357:1364 (<1%)	view
7	x 2	luigi/contrib luigi/contrib	esindex.py esindex.py	151:157 (3%) 385:391 (3%)	view
7	x 2	luigi/contrib luigi/contrib	opener.py opener.py	208:215 (6%) 233:240 (6%)	view
7	x 2	luigi/contrib luigi/contrib	redshift.py redshift.py	411:422 (1%) 725:736 (1%)	view
6	x 2	luigi/static/visualiser/js luigi/static/visualiser/js	visualiserApp.js visualiserApp.js	1357:1363 (<1%) 1369:1375 (<1%)	view
6	x 2	luigi/contrib/hdfs luigi/contrib/hdfs	hadoopcli_clients.py hadoopcli_clients.py	102:109 (3%) 240:247 (3%)	view
6	x 2	luigi/contrib luigi/contrib	azureblob.py s3.py	253:260 (3%) 603:610 (1%)	view
6	x 2	luigi/contrib luigi/contrib	dataproc.py dataproc.py	61:67 (3%) 81:87 (3%)	view
6	x 2	luigi/contrib luigi/contrib	redshift.py redshift.py	411:421 (1%) 656:667 (1%)	view
6	x 2	luigi/contrib luigi/contrib	azureblob.py dropbox.py	253:260 (3%) 230:237 (3%)	view
6	x 2	luigi/contrib luigi/contrib	hadoop.py pig.py	225:232 (<1%) 181:188 (5%)	view
6	x 2	luigi/contrib luigi/contrib	redshift.py redshift.py	656:667 (1%) 725:735 (1%)	view
6	x 2	luigi/contrib/hdfs luigi/contrib/hdfs	hadoopcli_clients.py hadoopcli_clients.py	166:171 (3%) 230:235 (3%)	view
6	x 2	luigi/contrib luigi	azureblob.py format.py	253:260 (3%) 161:168 (1%)	view
6	x 2	luigi luigi	server.py server.py	132:138 (2%) 149:155 (2%)	view
6	x 2	luigi/static/visualiser/js luigi/static/visualiser/js	visualiserApp.js visualiserApp.js	1437:1442 (<1%) 1477:1482 (<1%)	view
6	x 2	luigi/contrib luigi/contrib	dropbox.py dropbox.py	101:107 (3%) 133:139 (3%)	view
6	x 2	luigi/contrib luigi/contrib	mssqldb.py mysqldb.py	71:85 (9%) 71:85 (4%)	view
6	x 2	luigi/static/visualiser/js luigi/static/visualiser/js	visualiserApp.js visualiserApp.js	1342:1348 (<1%) 1369:1375 (<1%)	view
6	x 2	luigi/contrib luigi/contrib	dropbox.py s3.py	230:237 (3%) 603:610 (1%)	view
6	x 2	luigi/contrib luigi	s3.py format.py	603:610 (1%) 161:168 (1%)	view
6	x 2	luigi/contrib luigi	dropbox.py format.py	230:237 (3%) 161:168 (1%)	view
6	x 2	luigi/static/visualiser luigi/static/visualiser	index.html index.html	40:45 (1%) 241:246 (1%)	view
6	x 2	luigi/contrib luigi/contrib	ssh.py ssh.py	169:174 (2%) 190:195 (2%)	view
6	x 2	luigi/contrib luigi/contrib	redshift.py redshift.py	656:667 (1%) 799:809 (1%)	view