1 line
9.8 KiB
JSON
1 line
9.8 KiB
JSON
{"url": "http://mondego.ics.uci.edu/projects/clonedetection/", "content": "\n<!-- saved from url=(0032)http://www.ics.uci.edu/~vpsaini/ -->\n<html><head><meta charset='utf-8'>\n<title>Code Clone Detection</title> \n<meta name=\"description\" content=\"Yelp dataset challenge\">\n<meta name=\"keywords\" content=\"Yelp, challenge, data mining\">\n<style>\ncode {background-color:#A9A9A9;}\nA { COLOR: #62021; TEXT-DECORATION: none ; }\na:visited{ color: blue }\nA:hover { TEXT-DECORATION: underline }\n.lowspace { min-height: 5}\n.midspace { min-height: 10}\n.highspace { min-height: 15}\n.media {width:459;height:358;}\n.left{float:left;}\n.right{float:right;}\n.photo{height: 309;\n\twidth: 430;\n\tpadding-left: 57;\n\tpadding-bottom: 25;}\n\ttable {\n\t\tpadding-left: 20;\n\t\tpadding-right: 20;\n\t}\n\ttd {\n\t\ttext-align: justify;\n\t\tfont-size: 110%;\n\t\tline-height: 150%;\n\t\tmargin-bottom: 48px;\n\t\tmargin-top: -22px;\n\t\tfont-family: Georgia;\n\t}\n\tdivimg {\n\t\ttext-align: center;\n\t\tfont-size: 110%;\n\t\tline-height: 150%;\n\t\tmargin-bottom: 48px;\n\t\tmargin-top: -22px;\n\t\tfont-family: Georgia;\n\t}\n\t.mondego{width:300px;\n\t\theight:110px;\n\t\tbackground: transparent url('http://mondego.ics.uci.edu/img/mondego-banner.png') -0px -0px no-repeat;}\n\t\t</style>\n\n\t\t<style type=\"text/css\"></style><style type=\"text/css\"></style></head>\n\t\t<body link=\"blue\" alink=\"blue\" vlink=\"violet\">\n\t\t\t<script>\n\t\t\t (function(i,s,o,g,r,a,m){i['GoogleAnalyticsObject']=r;i[r]=i[r]||function(){\n\t\t\t (i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new Date();a=s.createElement(o),\n\t\t\t m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m)\n\t\t\t })(window,document,'script','//www.google-analytics.com/analytics.js','ga');\n\n\t\t\t ga('create', 'UA-41064162-1', 'uci.edu');\n\t\t\t ga('send', 'pageview');\n\n\t\t\t</script>\n\t\t\t\t\t\t\n\n\t\t\t<table>\n\t\t\t\t<tbody><tr>\n <td class=\"mondego\"></td>\n <td width=\"20\">\n\t\t\t</td><td><h2>Scaling Token-Based Code Clone Detection</h2>\n\n\t\t\t<i>\n\t\t\t\tThe Team<br>\n\t\t\t\t<a href =\"http://www.ics.uci.edu/~hsajnani/\">Hitesh Sajnani</a>, <a href=\"http://www.linkedin.com/in/sainivaibhav\">Vaibhav Saini</a>, <a href=\"http://www.ics.uci.edu/~lopes/\">Cristina Lopes</a> <br>\n\t\t\t</i>\n\t\t</div></div></td>\n\t\t</tr></tbody></table>\n\t\t<table>\n\t\t\t<tbody>\n\t\t\t\t<tr><td><div id=abstract><br><h3>Abstract</h3>\nWe propose a new token-based approach for large scale code clone detection which is based on a filtering heuristic that reduces the number of token comparisons when the two code blocks are compared. We also present a MapReduce based parallel algorithm that uses the filtering heuristic and scales to thousands of projects. The filtering heuristic is generic and can also be used in conjunction with other token-based approaches. In that context, we demonstrate how it can increase the retrieval speed and decrease the memory usage of the index-based approaches.\n<br>\nIn our experiments on 36 open source Java projects, we found that: (i) filtering reduces token comparisons by a factor of 10, and thus increasing\nthe speed of clone detection by a factor of 1.5; (ii) the speed-up and scale-up of the parallel approach using filtering is near-linear on a cluster of\n2-32 nodes for 150-2800 projects; and (iii) filtering decreases the memory usage of index-based approach by half and the search time by a factor of 5.\u00e2\u20ac\u2039</div>\t\t\t\t\n</tr>\n<tr>\n\t<td><br><div id=repexp>\n\t\t<h3>Replicating the Experiment</h3>\n\t\t<h4>System Requirements</h4>\n\t\tYou need a machine with at least 12GB ram running ubuntu or mac-osx, with Java-1.6 or higher installed. Please note that you need system with higher ram because the experiment involves very large subject systems. However, if you are running this on smaller systems (see Tools below), there is no such constraint.\n\t\t<h4>Steps to replicate the experiment:</h4>\n\t\t<ol>\n\t\t<li>\n\t\tClick<a href=\"files/dist/dist.zip\"> here</a> to download the distribution (dist.zip). \n\t\t</li>\n\n\t\t<li>\n unzip dist.zip. Navigate to dist/ using terminal\n </li>\n\n\t\t<li>\n execute the command <code style=\"background-color:#A9A9A9\">./runexp.sh 1 </code>\n\t\t    Please Note: this will run the experiment 1 time. To run it <i>n</i> times, change the command to <code style=\"background-color:#A9A9A9\">./runexp.sh n</code>\n </li>\n\t\t</ol>\n\tYou can see the generated output in the ./output folder. Files with extension <i>.txt</i> will have computed clones and the files with <i>.csv</i> extension will have the performance analysis result. \n</div>\n<div id=tool>\t\t\n<h3>Tools</h3>\nIn order to run the tool on any arbitrary project, please follow the steps below:\n<br><br>\n<Strong>A.</Strong> Generating the input file of the project for which you want to detect clones\n<ol>\n<li>Click <a href=files/tools/inputgen/ast.zip>here</a> to download input generator for the code clone detector (ast.zip).</li>\n<li>Unzip ast.zip and import the project ast in your eclipse workspace.</li>\n<li>Run it as an \"Eclipse Application\". This should open another eclipse instance where you will import the projects for which you want to generate the input file.</li> \n<li>After importing the project in the workspace of the new eclipse instance, click on the \"Sample Menu\" in the top menu bar and then click on the \"Sample command\" to run. This should generate the output (desired input file) in the path specified by variable \"outputdirPath\".</li>\n<li>Please note that you will have to change the location of output directory on line 61 of SampleHandler.java.<code>this.outputdirPath = \"/Users/vaibhavsaini/Documents/codetime/repo/ast/output/\";</code> to your desired output directory.</li>\n<li>The generated input file name will be of the format: <ProjectName>-clone-INPUT.txt. For example, if your project name is jython, then the generated input file name should be jython-clone-INPUT.txt</li>\n</ol>\n<br>\n<strong>B</strong>. Running the clone detection tool on the generated input file\n<ol>\n<li>Click<a href=\"files/dist/tool.zip\"> here</a> to download the CloneDetector (tool.zip).</li>\n<li>Unzip tool.zip and navigate to tool/ using terminal</li>\n<li>Copy the input file generated above (<ProjectName>-clone-INPUT.txt) into input/dataset directory.</li>\n<li>Open cd.sh, and assign <ProjectName> as value to the variable <code> arrayname</code> (line #5). For example, If your generated input file is jython-clone-INPUT.txt, line #5 should be <code>arrayname=(jython)</code></li>\n<li>Execute the command <code>./cd.sh</code></li>\n</ol>\n<br>\n<strong>C.</strong> Generated output\n<ol>\n<li>The generated output will be in the ./output folder. </li>\n<li>Files with extension <i>.txt</i> will have the computed clones and the files with <i>.csv</i> extension will have the time taken to detect clones</li>\n</div>\n\t\t<h3>Data</h3>\nThe table below describes all the subject systems, and the corresponding output. This data was used to calculate and report numbers in the paper. \nColumn 1 has links to the source code of each subject system. Column 2 is the input file that is generated by running INPUTGEN tool on the subject systems. \nThis input file is in turn used by NCCF and FCCD to compute clones. Column 4 has links to the computed clones. Since both the tools produced exactly the same output, you will see only one file per subject system. Column 4 has the final analysis results - runtime for tool, and total token comparisons done. These numbers are different for both the tools, so you will see two analysis files per project - one produced by NCCF, and another by FCCD. \n<table border=\"1\" bordercolor=\"#000000\" style=\"background-color:#FAFAF8;padding-left:0;padding-right:0\" width=\"51%\" cellpadding=\"3\" cellspacing=\"1\">\n\t<tr>\n\t\t<th>Subject System\n\t\t<th>Generated Input for Tool\n\t\t<th>Clones Detected\n\t\t<th>Analysis\n\t</tr>\n\t<tr>\n\t\t<td><a href=http://supergsego.com/apache/ant/source/apache-ant-1.9.2-src.zip>Ant-1.9.2</a></td>\n\t\t<td><a href=files/input/ant.zip>ant.zip</a></td>\n\t\t<td><a href=files/output/ant-clones.zip>ant-clones.zip</a></td>\n\t\t<td><a href=files/output/ant-analysis.zip>ant-analysis.zip</a></td>\n\t<tr>\n\t\t<td><a href=http://www.apache.org/dyn/closer.cgi/cocoon/2.2/cocoon-2.2.0.zip>Cocoon-2.2.0</a></td>\n\t\t<td><a href=files/input/cocoon.zip>cocoon.zip</a></td>\n\t\t<td><a href=files/output/cocoon-clones.zip>cocoon-clones.zip</a></td>\n\t\t<td><a href=files/output/cocoon-analysis.zip>cocoon-analysis.zip</a></td>\n\n\t</tr>\n\t<tr>\n\t\t<td><a href=http://mirror.tcpdiag.net/apache/maven/maven-3/3.0.5/source/apache-maven-3.0.5-src.zip>Maven-3.0.5</a></td>\n\t\t<td><a href=files/input/maven.zip>maven.zip</a></td>\n\t\t<td><a href=files/output/maven-clones.zip>maven-clones.zip</a></td>\n\t\t<td><a href=files/output/maven-analysis.zip>maven-analysis.zip</a></td>\n\t</tr>\n\t<tr>\n\t\t<td><a href=http://archive.apache.org/dist/lucene/java/4.4.0/>Lucene-4.4.0</a></td>\n\t\t<td><a href=files/input/lucene.zip>lucene.zip</a></td>\n\t\t<td><a href=files/output/lucene-clones.zip>lucene-clones.zip</a></td>\n\t\t<td><a href=files/output/lucene-analysis.zip>lucene-analysis.zip</a></td>\n\t</tr>\n\t<tr>\n\t\t<td><a href=http://svn.apache.org/repos/asf/hadoop/common/trunk/>Hadoop-Rev:1531458</a></td>\n\t\t<td><a href=files/input/hadoop.zip>hadoop.zip</a></td>\n\t\t<td><a href=files/output/hadoop-clones.zip>hadoop-clones.zip</a></td>\n\t\t<td><a href=files/output/hadoop-analysis.zip>hadoop-analysis.zip</a></td>\n\t</tr>\n</table>\t\n\t\t<!--<h3>Validated Clones</h3>\n\t\t<a href=files/validation/validated-clones.txt.>validated-clones.txt</a>\n!-->\n</td>\n</tr>\n</tr>\n</tbody>\n</table>\n</body></html>\n", "encoding": "Windows-1252"} |