tag:blogger.com,1999:blog-24884907461368307462023-11-17T01:18:55.859+07:00IT Experienceพี่นัทhttp://www.blogger.com/profile/14291406590594822628noreply@blogger.comBlogger56125tag:blogger.com,1999:blog-2488490746136830746.post-29412784556055404572017-03-26T21:33:00.001+07:002017-03-29T22:23:33.136+07:00mongoDB Backup and Restore<br />
<br />
Install MongoDB Community Edition on Ubuntu (Optional)<br />
<br />
<ol>
<li>Import the public key used by the package management system.<br /><span style="color: blue;">sudo apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv 0C49F3730359A14518585931BC711F9BA15703C6</span></li>
<li>Create a list file for MongoDB.<br />(Trusty) <span style="color: blue;">echo "deb [ arch=amd64 ] http://repo.mongodb.org/apt/ubuntu trusty/mongodb-org/3.4 multiverse" | sudo tee /etc/apt/sources.list.d/mongodb-org-3.4.list</span><br />(Xenial) <span style="color: blue;">echo "deb [ arch=amd64,arm64 ] http://repo.mongodb.org/apt/ubuntu xenial/mongodb-org/3.4 multiverse" | sudo tee /etc/apt/sources.list.d/mongodb-org-3.4.list</span></li>
<li>Reload local package database<br /><span style="color: blue;">sudo apt-get update</span></li>
<li>Install the MongoDB packages (Client only)<br /><span style="color: blue;">sudo apt-get install -y mongodb-org-shell mongodb-org-tools</span></li>
</ol>
<br />
<br />
Backup (Source database from INET Cloud)<br />
<!-- HTML generated using hilite.me --><br />
<div style="background: #f0f0f0; border-width: 0.1em 0.1em 0.1em 0.8em; border: solid gray; overflow: auto; padding: 0.2em 0.6em; width: auto;">
<pre style="line-height: 125%; margin: 0;">mongodump --host<span style="color: #666666;">=</span>172.17.0.3 --port<span style="color: #666666;">=</span>27017 <span style="color: #4070a0; font-weight: bold;">\</span>
--username<span style="color: #666666;">=</span>nutt --password<span style="color: #666666;">=</span>P@ssw0rd1 <span style="color: #4070a0; font-weight: bold;">\</span>
--authenticationDatabase<span style="color: #666666;">=</span>admin <span style="color: #4070a0; font-weight: bold;">\</span>
--db<span style="color: #666666;">=</span>settrade <span style="color: #4070a0; font-weight: bold;">\</span>
--archive<span style="color: #666666;">=</span>/home/ubuntu/backup/settrade.dmp <span style="color: #4070a0; font-weight: bold;">\</span>
--gzip <span style="color: #4070a0; font-weight: bold;">\</span>
--dumpDbUsersAndRoles <span style="color: #4070a0; font-weight: bold;">\</span>
--verbose<span style="color: #666666;">=</span>5
</pre>
</div>
<br />
<br />
Restore (Database on Windows as destination)<br />
<br />
<div style="background: #f0f0f0; border-width: 0.1em 0.1em 0.1em 0.8em; border: solid gray; overflow: auto; padding: 0.2em 0.6em; width: auto;">
<pre style="line-height: 125%; margin: 0;">mongorestore /host:127.0.0.1 /port:27017 /db:settrade /gzip /drop /restoreDbUsersAndRoles /archive:settrade.dmp
</pre>
</div>
When run montorestore command on Ubuntu, an arguments changes as below:<br />
<span style="color: blue;">mongorestore --host=172.17.0.1 --port=27017 --db=settrade --gzip --drop --restoreDbUsersAndRoles --archive=settrade.dmp</span><br />
<br />
Check number of objects in databases (ensure data structure correctly should run <span style="color: blue;">db.collection.validate({full: true}) </span>before counting)<br />
<br />
<div style="background: #f0f0f0; border-width: 0.1em 0.1em 0.1em 0.8em; border: solid gray; overflow: auto; padding: 0.2em 0.6em; width: auto;">
<pre style="line-height: 125%; margin: 0;">use settrade
var collections = db.getCollectionNames();
print('Collections inside the db:');
for(var i = 0,c = 0; i < collections.length; i++){
var name = collections[i];
if(name.substr(0, 6) != 'system')
print(name + ' - ' + db[name].count() + ' records');
c = c + db[name].count()
}
print("All objects in db: ",c)
</pre>
</div>
<br />
<br />พี่นัทhttp://www.blogger.com/profile/14291406590594822628noreply@blogger.com0tag:blogger.com,1999:blog-2488490746136830746.post-39451379339406642532017-03-13T22:28:00.000+07:002017-03-13T22:28:07.339+07:00Request and install Let's Encrypt certificates to Microsoft IIS<br />
<br />
Follow an instruction from orginal site <a href="https://github.com/ebekker/ACMESharp/wiki/Quick-Start" target="_blank">https://github.com/ebekker/ACMESharp/wiki/Quick-Start</a><br />
<br />
<br />
<!-- HTML generated using hilite.me --><br />
<div style="background: #202020; border-width: 0.1em 0.1em 0.1em 0.8em; border: solid gray; overflow: auto; padding: 0.2em 0.6em; width: auto;">
<pre style="line-height: 125%; margin: 0;"><span style="color: #d0d0d0;">Windows</span> <span style="color: #d0d0d0;">PowerShell</span>
<span style="color: #d0d0d0;">Copyright</span> <span style="color: #d0d0d0;">(C)</span> <span style="color: #d0d0d0;">2014</span> <span style="color: #d0d0d0;">Microsoft</span> <span style="color: #d0d0d0;">Corporation.</span> <span style="color: #d0d0d0;">All</span> <span style="color: #d0d0d0;">rights</span> <span style="color: #d0d0d0;">reserved.</span>
<span style="color: #d0d0d0;">PS</span> <span style="color: #d0d0d0;">C</span><span style="background-color: #e3d2d2; color: #a61717;">:</span><span style="color: #d0d0d0;">\Users\Administrator></span> <span style="color: #24909d;">Import-Module</span> <span style="color: #d0d0d0;">ACMESharp</span>
<span style="color: #d0d0d0;">PS</span> <span style="color: #d0d0d0;">C</span><span style="background-color: #e3d2d2; color: #a61717;">:</span><span style="color: #d0d0d0;">\Users\Administrator></span> <span style="color: #d0d0d0;">Initialize-ACMEVault</span>
<span style="color: #d0d0d0;">Initialize-ACMEVault</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">Vault</span> <span style="color: #d0d0d0;">root</span> <span style="color: #d0d0d0;">path</span> <span style="color: #d0d0d0;">contains</span> <span style="color: #d0d0d0;">existing</span> <span style="color: #d0d0d0;">vault</span> <span style="color: #d0d0d0;">data</span>
<span style="color: #d0d0d0;">At</span> <span style="color: #d0d0d0;">line</span><span style="background-color: #e3d2d2; color: #a61717;">:</span><span style="color: #d0d0d0;">1</span> <span style="color: #d0d0d0;">char</span><span style="background-color: #e3d2d2; color: #a61717;">:</span><span style="color: #d0d0d0;">1</span>
<span style="color: #d0d0d0;">+</span> <span style="color: #d0d0d0;">Initialize-ACMEVault</span>
<span style="color: #d0d0d0;">+</span> <span style="color: #d0d0d0;">~~~~~~~~~~~~~~~~~~~~</span>
<span style="color: #d0d0d0;">+</span> <span style="color: #d0d0d0;">CategoryInfo</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">NotSpecified</span><span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">(</span><span style="background-color: #e3d2d2; color: #a61717;">:</span><span style="color: #d0d0d0;">)</span> <span style="color: #d0d0d0;">[Initialize-ACMEVault],</span> <span style="color: #d0d0d0;">Exception</span>
<span style="color: #d0d0d0;">+</span> <span style="color: #d0d0d0;">FullyQualifiedErrorId</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">System.Exception,ACMESharp.POSH.InitializeVault</span>
<span style="color: #d0d0d0;">PS</span> <span style="color: #d0d0d0;">C</span><span style="background-color: #e3d2d2; color: #a61717;">:</span><span style="color: #d0d0d0;">\Users\Administrator></span> <span style="color: #d0d0d0;">mkdir</span> <span style="color: #d0d0d0;">c</span><span style="background-color: #e3d2d2; color: #a61717;">:</span><span style="color: #d0d0d0;">\Vault</span>
<span style="color: #d0d0d0;">Directory</span><span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">C</span><span style="background-color: #e3d2d2; color: #a61717;">:</span><span style="color: #d0d0d0;">\</span>
<span style="color: #d0d0d0;">Mode</span> <span style="color: #d0d0d0;">LastWriteTime</span> <span style="color: #d0d0d0;">Length</span> <span style="color: #d0d0d0;">Name</span>
<span style="color: #d0d0d0;">----</span> <span style="color: #d0d0d0;">-------------</span> <span style="color: #d0d0d0;">------</span> <span style="color: #d0d0d0;">----</span>
<span style="color: #d0d0d0;">d----</span> <span style="color: #d0d0d0;">3/9/2017</span> <span style="color: #d0d0d0;">3</span><span style="background-color: #e3d2d2; color: #a61717;">:</span><span style="color: #d0d0d0;">59</span> <span style="color: #d0d0d0;">PM</span> <span style="color: #d0d0d0;">Vault</span>
<span style="color: #d0d0d0;">PS</span> <span style="color: #d0d0d0;">C</span><span style="background-color: #e3d2d2; color: #a61717;">:</span><span style="color: #d0d0d0;">\Users\Administrator></span> <span style="color: #d0d0d0;">cd</span> <span style="color: #d0d0d0;">C</span><span style="background-color: #e3d2d2; color: #a61717;">:</span><span style="color: #d0d0d0;">\Vault</span>
<span style="color: #d0d0d0;">PS</span> <span style="color: #d0d0d0;">C</span><span style="background-color: #e3d2d2; color: #a61717;">:</span><span style="color: #d0d0d0;">\Vault></span> <span style="color: #24909d;">Import-Module</span> <span style="color: #d0d0d0;">ACMEPowerShell</span>
<span style="color: #24909d;">Import-Module</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">The</span> <span style="color: #d0d0d0;">specified</span> <span style="color: #d0d0d0;">module</span> <span style="color: #ed9d13;">'ACMEPowerShell'</span> <span style="color: #d0d0d0;">was</span> <span style="color: #d0d0d0;">not</span> <span style="color: #d0d0d0;">loaded</span> <span style="color: #d0d0d0;">because</span> <span style="color: #d0d0d0;">no</span> <span style="color: #d0d0d0;">valid</span> <span style="color: #d0d0d0;">module</span> <span style="color: #d0d0d0;">file</span> <span style="color: #d0d0d0;">was</span> <span style="color: #d0d0d0;">found</span> <span style="color: #6ab825; font-weight: bold;">in</span> <span style="color: #d0d0d0;">any</span>
<span style="color: #d0d0d0;">module</span> <span style="color: #d0d0d0;">directory.</span>
<span style="color: #d0d0d0;">At</span> <span style="color: #d0d0d0;">line</span><span style="background-color: #e3d2d2; color: #a61717;">:</span><span style="color: #d0d0d0;">1</span> <span style="color: #d0d0d0;">char</span><span style="background-color: #e3d2d2; color: #a61717;">:</span><span style="color: #d0d0d0;">1</span>
<span style="color: #d0d0d0;">+</span> <span style="color: #24909d;">Import-Module</span> <span style="color: #d0d0d0;">ACMEPowerShell</span>
<span style="color: #d0d0d0;">+</span> <span style="color: #d0d0d0;">~~~~~~~~~~~~~~~~~~~~~~~~~~~~</span>
<span style="color: #d0d0d0;">+</span> <span style="color: #d0d0d0;">CategoryInfo</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">ResourceUnavailable</span><span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">(ACMEPowerShell</span><span style="background-color: #e3d2d2; color: #a61717;">:</span><span style="color: #d0d0d0;">String)</span> <span style="color: #d0d0d0;">[</span><span style="color: #24909d;">Import-Module</span><span style="color: #d0d0d0;">],</span> <span style="color: #d0d0d0;">FileNotFoundException</span>
<span style="color: #d0d0d0;">+</span> <span style="color: #d0d0d0;">FullyQualifiedErrorId</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">Modules_ModuleNotFound,Microsoft.PowerShell.Commands.ImportModuleCommand</span>
<span style="color: #d0d0d0;">PS</span> <span style="color: #d0d0d0;">C</span><span style="background-color: #e3d2d2; color: #a61717;">:</span><span style="color: #d0d0d0;">\Vault></span> <span style="color: #d0d0d0;">dir</span>
<span style="color: #d0d0d0;">Directory</span><span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">C</span><span style="background-color: #e3d2d2; color: #a61717;">:</span><span style="color: #d0d0d0;">\Vault</span>
<span style="color: #d0d0d0;">Mode</span> <span style="color: #d0d0d0;">LastWriteTime</span> <span style="color: #d0d0d0;">Length</span> <span style="color: #d0d0d0;">Name</span>
<span style="color: #d0d0d0;">----</span> <span style="color: #d0d0d0;">-------------</span> <span style="color: #d0d0d0;">------</span> <span style="color: #d0d0d0;">----</span>
<span style="color: #d0d0d0;">-a---</span> <span style="color: #d0d0d0;">3/9/2017</span> <span style="color: #d0d0d0;">4</span><span style="background-color: #e3d2d2; color: #a61717;">:</span><span style="color: #d0d0d0;">02</span> <span style="color: #d0d0d0;">PM</span> <span style="color: #d0d0d0;">145</span> <span style="color: #d0d0d0;">ACMEPowerShell-IIS.psd1</span>
<span style="color: #d0d0d0;">PS</span> <span style="color: #d0d0d0;">C</span><span style="background-color: #e3d2d2; color: #a61717;">:</span><span style="color: #d0d0d0;">\Vault></span> <span style="color: #d0d0d0;">.\ACMEPowerShell-IIS.psd1</span>
<span style="color: #d0d0d0;">PS</span> <span style="color: #d0d0d0;">C</span><span style="background-color: #e3d2d2; color: #a61717;">:</span><span style="color: #d0d0d0;">\Vault></span> <span style="color: #24909d;">Import-Module</span> <span style="color: #d0d0d0;">.\ACMEPowerShell-IIS.psd1</span>
<span style="color: #24909d;">Import-Module</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">The</span> <span style="color: #d0d0d0;">module</span> <span style="color: #d0d0d0;">to</span> <span style="color: #6ab825; font-weight: bold;">process</span> <span style="color: #ed9d13;">'ACMEPowerShell-IIS'</span><span style="color: #d0d0d0;">,</span> <span style="color: #d0d0d0;">listed</span> <span style="color: #6ab825; font-weight: bold;">in</span> <span style="color: #d0d0d0;">field</span> <span style="color: #ed9d13;">'ModuleToProcess/RootModule'</span> <span style="color: #d0d0d0;">of</span> <span style="color: #d0d0d0;">module</span>
<span style="color: #d0d0d0;">manifest</span> <span style="color: #ed9d13;">'C:\Vault\ACMEPowerShell-IIS.psd1'</span> <span style="color: #d0d0d0;">was</span> <span style="color: #d0d0d0;">not</span> <span style="color: #d0d0d0;">processed</span> <span style="color: #d0d0d0;">because</span> <span style="color: #d0d0d0;">no</span> <span style="color: #d0d0d0;">valid</span> <span style="color: #d0d0d0;">module</span> <span style="color: #d0d0d0;">was</span> <span style="color: #d0d0d0;">found</span> <span style="color: #6ab825; font-weight: bold;">in</span> <span style="color: #d0d0d0;">any</span> <span style="color: #d0d0d0;">module</span>
<span style="color: #d0d0d0;">directory.</span>
<span style="color: #d0d0d0;">At</span> <span style="color: #d0d0d0;">line</span><span style="background-color: #e3d2d2; color: #a61717;">:</span><span style="color: #d0d0d0;">1</span> <span style="color: #d0d0d0;">char</span><span style="background-color: #e3d2d2; color: #a61717;">:</span><span style="color: #d0d0d0;">1</span>
<span style="color: #d0d0d0;">+</span> <span style="color: #24909d;">Import-Module</span> <span style="color: #d0d0d0;">.\ACMEPowerShell-IIS.psd1</span>
<span style="color: #d0d0d0;">+</span> <span style="color: #d0d0d0;">~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~</span>
<span style="color: #d0d0d0;">+</span> <span style="color: #d0d0d0;">CategoryInfo</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">ResourceUnavailable</span><span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">(ACMEPowerShell-IIS</span><span style="background-color: #e3d2d2; color: #a61717;">:</span><span style="color: #d0d0d0;">String)</span> <span style="color: #d0d0d0;">[</span><span style="color: #24909d;">Import-Module</span><span style="color: #d0d0d0;">],</span> <span style="color: #d0d0d0;">PSInvalidOperationExce</span>
<span style="color: #d0d0d0;">ption</span>
<span style="color: #d0d0d0;">+</span> <span style="color: #d0d0d0;">FullyQualifiedErrorId</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">Modules_ModuleFileNotFound,Microsoft.PowerShell.Commands.ImportModuleCommand</span>
<span style="color: #d0d0d0;">PS</span> <span style="color: #d0d0d0;">C</span><span style="background-color: #e3d2d2; color: #a61717;">:</span><span style="color: #d0d0d0;">\Vault></span> <span style="color: #d0d0d0;">Save-Module</span> <span style="color: #d0d0d0;">-Name</span> <span style="color: #d0d0d0;">ACMESharp</span> <span style="color: #d0d0d0;">-Path</span> <span style="color: #d0d0d0;">C</span><span style="background-color: #e3d2d2; color: #a61717;">:</span><span style="color: #d0d0d0;">\ACME</span>
<span style="color: #d0d0d0;">Save-Module</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">Cannot</span> <span style="color: #d0d0d0;">find</span> <span style="color: #d0d0d0;">the</span> <span style="color: #d0d0d0;">path</span> <span style="color: #ed9d13;">'C:\ACME'</span> <span style="color: #d0d0d0;">because</span> <span style="color: #d0d0d0;">it</span> <span style="color: #d0d0d0;">does</span> <span style="color: #d0d0d0;">not</span> <span style="color: #d0d0d0;">exist.</span>
<span style="color: #d0d0d0;">At</span> <span style="color: #d0d0d0;">line</span><span style="background-color: #e3d2d2; color: #a61717;">:</span><span style="color: #d0d0d0;">1</span> <span style="color: #d0d0d0;">char</span><span style="background-color: #e3d2d2; color: #a61717;">:</span><span style="color: #d0d0d0;">1</span>
<span style="color: #d0d0d0;">+</span> <span style="color: #d0d0d0;">Save-Module</span> <span style="color: #d0d0d0;">-Name</span> <span style="color: #d0d0d0;">ACMESharp</span> <span style="color: #d0d0d0;">-Path</span> <span style="color: #d0d0d0;">C</span><span style="background-color: #e3d2d2; color: #a61717;">:</span><span style="color: #d0d0d0;">\ACME</span>
<span style="color: #d0d0d0;">+</span> <span style="color: #d0d0d0;">~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~</span>
<span style="color: #d0d0d0;">+</span> <span style="color: #d0d0d0;">CategoryInfo</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">InvalidOperation</span><span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">(</span><span style="background-color: #e3d2d2; color: #a61717;">:</span><span style="color: #d0d0d0;">)</span> <span style="color: #d0d0d0;">[Save-Module],</span> <span style="color: #d0d0d0;">InvalidOperationException</span>
<span style="color: #d0d0d0;">+</span> <span style="color: #d0d0d0;">FullyQualifiedErrorId</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">PathNotFound,Save-Module</span>
<span style="color: #d0d0d0;">PS</span> <span style="color: #d0d0d0;">C</span><span style="background-color: #e3d2d2; color: #a61717;">:</span><span style="color: #d0d0d0;">\Vault></span> <span style="color: #d0d0d0;">Save-Module</span> <span style="color: #d0d0d0;">-Name</span> <span style="color: #d0d0d0;">ACMESharp</span> <span style="color: #d0d0d0;">-Path</span> <span style="color: #d0d0d0;">C</span><span style="background-color: #e3d2d2; color: #a61717;">:</span><span style="color: #d0d0d0;">\ACME</span>
<span style="color: #d0d0d0;">PS</span> <span style="color: #d0d0d0;">C</span><span style="background-color: #e3d2d2; color: #a61717;">:</span><span style="color: #d0d0d0;">\Vault></span> <span style="color: #d0d0d0;">Install-Module</span> <span style="color: #d0d0d0;">-Name</span> <span style="color: #d0d0d0;">ACMESharp</span>
<span style="color: #d0d0d0;">PS</span> <span style="color: #d0d0d0;">C</span><span style="background-color: #e3d2d2; color: #a61717;">:</span><span style="color: #d0d0d0;">\Vault></span> <span style="color: #24909d;">Import-Module</span> <span style="color: #d0d0d0;">ACMEPowerShell</span>
<span style="color: #24909d;">Import-Module</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">The</span> <span style="color: #d0d0d0;">specified</span> <span style="color: #d0d0d0;">module</span> <span style="color: #ed9d13;">'ACMEPowerShell'</span> <span style="color: #d0d0d0;">was</span> <span style="color: #d0d0d0;">not</span> <span style="color: #d0d0d0;">loaded</span> <span style="color: #d0d0d0;">because</span> <span style="color: #d0d0d0;">no</span> <span style="color: #d0d0d0;">valid</span> <span style="color: #d0d0d0;">module</span> <span style="color: #d0d0d0;">file</span> <span style="color: #d0d0d0;">was</span> <span style="color: #d0d0d0;">found</span> <span style="color: #6ab825; font-weight: bold;">in</span> <span style="color: #d0d0d0;">any</span>
<span style="color: #d0d0d0;">module</span> <span style="color: #d0d0d0;">directory.</span>
<span style="color: #d0d0d0;">At</span> <span style="color: #d0d0d0;">line</span><span style="background-color: #e3d2d2; color: #a61717;">:</span><span style="color: #d0d0d0;">1</span> <span style="color: #d0d0d0;">char</span><span style="background-color: #e3d2d2; color: #a61717;">:</span><span style="color: #d0d0d0;">1</span>
<span style="color: #d0d0d0;">+</span> <span style="color: #24909d;">Import-Module</span> <span style="color: #d0d0d0;">ACMEPowerShell</span>
<span style="color: #d0d0d0;">+</span> <span style="color: #d0d0d0;">~~~~~~~~~~~~~~~~~~~~~~~~~~~~</span>
<span style="color: #d0d0d0;">+</span> <span style="color: #d0d0d0;">CategoryInfo</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">ResourceUnavailable</span><span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">(ACMEPowerShell</span><span style="background-color: #e3d2d2; color: #a61717;">:</span><span style="color: #d0d0d0;">String)</span> <span style="color: #d0d0d0;">[</span><span style="color: #24909d;">Import-Module</span><span style="color: #d0d0d0;">],</span> <span style="color: #d0d0d0;">FileNotFoundException</span>
<span style="color: #d0d0d0;">+</span> <span style="color: #d0d0d0;">FullyQualifiedErrorId</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">Modules_ModuleNotFound,Microsoft.PowerShell.Commands.ImportModuleCommand</span>
<span style="color: #d0d0d0;">PS</span> <span style="color: #d0d0d0;">C</span><span style="background-color: #e3d2d2; color: #a61717;">:</span><span style="color: #d0d0d0;">\Vault></span> <span style="color: #24909d;">Import-Module</span> <span style="color: #d0d0d0;">ACMESharp</span>
<span style="color: #d0d0d0;">PS</span> <span style="color: #d0d0d0;">C</span><span style="background-color: #e3d2d2; color: #a61717;">:</span><span style="color: #d0d0d0;">\Vault></span> <span style="color: #d0d0d0;">Initialize-ACMEVault</span> <span style="color: #d0d0d0;">-BaseURI</span> <span style="color: #d0d0d0;">https</span><span style="background-color: #e3d2d2; color: #a61717;">:</span><span style="color: #d0d0d0;">//acme-staging.api.letsencrypt.org/</span>
<span style="color: #d0d0d0;">Initialize-ACMEVault</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">Vault</span> <span style="color: #d0d0d0;">root</span> <span style="color: #d0d0d0;">path</span> <span style="color: #d0d0d0;">contains</span> <span style="color: #d0d0d0;">existing</span> <span style="color: #d0d0d0;">vault</span> <span style="color: #d0d0d0;">data</span>
<span style="color: #d0d0d0;">At</span> <span style="color: #d0d0d0;">line</span><span style="background-color: #e3d2d2; color: #a61717;">:</span><span style="color: #d0d0d0;">1</span> <span style="color: #d0d0d0;">char</span><span style="background-color: #e3d2d2; color: #a61717;">:</span><span style="color: #d0d0d0;">1</span>
<span style="color: #d0d0d0;">+</span> <span style="color: #d0d0d0;">Initialize-ACMEVault</span> <span style="color: #d0d0d0;">-BaseURI</span> <span style="color: #d0d0d0;">https</span><span style="background-color: #e3d2d2; color: #a61717;">:</span><span style="color: #d0d0d0;">//acme-staging.api.letsencrypt.org/</span>
<span style="color: #d0d0d0;">+</span> <span style="color: #d0d0d0;">~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~</span>
<span style="color: #d0d0d0;">+</span> <span style="color: #d0d0d0;">CategoryInfo</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">NotSpecified</span><span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">(</span><span style="background-color: #e3d2d2; color: #a61717;">:</span><span style="color: #d0d0d0;">)</span> <span style="color: #d0d0d0;">[Initialize-ACMEVault],</span> <span style="color: #d0d0d0;">Exception</span>
<span style="color: #d0d0d0;">+</span> <span style="color: #d0d0d0;">FullyQualifiedErrorId</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">System.Exception,ACMESharp.POSH.InitializeVault</span>
<span style="color: #d0d0d0;">PS</span> <span style="color: #d0d0d0;">C</span><span style="background-color: #e3d2d2; color: #a61717;">:</span><span style="color: #d0d0d0;">\Vault></span> <span style="color: #24909d;">New-ACMERegistration</span> <span style="color: #d0d0d0;">-Contacts</span> <span style="color: #d0d0d0;">mailto</span><span style="background-color: #e3d2d2; color: #a61717;">:</span><span style="color: #d0d0d0;">nutthaphon</span><span style="color: #40ffff;">@gmail</span><span style="color: #d0d0d0;">.com</span> <span style="color: #d0d0d0;">-AcceptTos</span>
<span style="color: #d0d0d0;">Contacts</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">{mailto</span><span style="background-color: #e3d2d2; color: #a61717;">:</span><span style="color: #d0d0d0;">nutthaphon</span><span style="color: #40ffff;">@gmail</span><span style="color: #d0d0d0;">.com}</span>
<span style="color: #d0d0d0;">PublicKey</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">{</span> <span style="color: #d0d0d0;">e</span> <span style="color: #d0d0d0;">=</span> <span style="color: #d0d0d0;">AQAB,</span> <span style="color: #d0d0d0;">kty</span> <span style="color: #d0d0d0;">=</span> <span style="color: #d0d0d0;">RSA,</span> <span style="color: #d0d0d0;">n</span> <span style="color: #d0d0d0;">=</span> <span style="color: #d0d0d0;">p0GDxWwNksxnglsbCJxwE6HtfYVTefb1dvsq7YovYlZtadoTqM4sWOzyIxUdBiHm5cqHGjE8</span>
<span style="color: #d0d0d0;">BcFbgHm9i1d5RCjRVGGoHQNM_D5LhtPjemXKtMEubbgjVDMinJONoK_COKcTs4L5MqbK20TKzdkGFNND69k29KbZWReXEeA8yVM</span>
<span style="color: #d0d0d0;">4H_aGI-FDkLO6ZxSWiuxkPchAgr1Sqj08pJR9X7qhJ2LX35Xgcuj0LkiuAkwnJuELZAhQgagK7_V9iibOBnJ9zwZlQaQ2aJoWlw</span>
<span style="color: #d0d0d0;">wJ51xP6enGSMDPZwpx5k4TJGcR46k3-NoFXfbAjv_MqzU6A9EwOmLBO5bhFkPwQ_25eaASPQ</span> <span style="color: #d0d0d0;">}</span>
<span style="color: #d0d0d0;">RecoveryKey</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span>
<span style="color: #d0d0d0;">RegistrationUri</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">https</span><span style="background-color: #e3d2d2; color: #a61717;">:</span><span style="color: #d0d0d0;">//acme-v01.api.letsencrypt.org/acme/reg/10593295</span>
<span style="color: #d0d0d0;">Links</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">{<https</span><span style="background-color: #e3d2d2; color: #a61717;">:</span><span style="color: #d0d0d0;">//acme-v01.api.letsencrypt.org/acme/</span><span style="color: #24909d;">new-authz</span><span style="color: #d0d0d0;">></span><span style="background-color: #e3d2d2; color: #a61717;">;</span><span style="color: #d0d0d0;">rel=</span><span style="color: #ed9d13;">"next"</span><span style="color: #d0d0d0;">,</span>
<span style="color: #d0d0d0;"><https</span><span style="background-color: #e3d2d2; color: #a61717;">:</span><span style="color: #d0d0d0;">//letsencrypt.org/documents/LE-SA-v1.1.1-August-1-2016.pdf></span><span style="background-color: #e3d2d2; color: #a61717;">;</span><span style="color: #d0d0d0;">rel=</span><span style="color: #ed9d13;">"terms-of-service"</span><span style="color: #d0d0d0;">}</span>
<span style="color: #d0d0d0;">TosLinkUri</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">https</span><span style="background-color: #e3d2d2; color: #a61717;">:</span><span style="color: #d0d0d0;">//letsencrypt.org/documents/LE-SA-v1.1.1-August-1-2016.pdf</span>
<span style="color: #d0d0d0;">TosAgreementUri</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">https</span><span style="background-color: #e3d2d2; color: #a61717;">:</span><span style="color: #d0d0d0;">//letsencrypt.org/documents/LE-SA-v1.1.1-August-1-2016.pdf</span>
<span style="color: #d0d0d0;">AuthorizationsUri</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span>
<span style="color: #d0d0d0;">CertificatesUri</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span>
<span style="color: #d0d0d0;">PS</span> <span style="color: #d0d0d0;">C</span><span style="background-color: #e3d2d2; color: #a61717;">:</span><span style="color: #d0d0d0;">\Vault></span> <span style="color: #d0d0d0;">hostname</span>
<span style="color: #d0d0d0;">mserver1</span>
<span style="color: #d0d0d0;">PS</span> <span style="color: #d0d0d0;">C</span><span style="background-color: #e3d2d2; color: #a61717;">:</span><span style="color: #d0d0d0;">\Vault></span> <span style="color: #24909d;">Import-Module</span> <span style="color: #d0d0d0;">ACMEPowerShell</span>
<span style="color: #24909d;">Import-Module</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">The</span> <span style="color: #d0d0d0;">specified</span> <span style="color: #d0d0d0;">module</span> <span style="color: #ed9d13;">'ACMEPowerShell'</span> <span style="color: #d0d0d0;">was</span> <span style="color: #d0d0d0;">not</span> <span style="color: #d0d0d0;">loaded</span> <span style="color: #d0d0d0;">because</span> <span style="color: #d0d0d0;">no</span> <span style="color: #d0d0d0;">valid</span> <span style="color: #d0d0d0;">module</span> <span style="color: #d0d0d0;">file</span> <span style="color: #d0d0d0;">was</span> <span style="color: #d0d0d0;">found</span> <span style="color: #6ab825; font-weight: bold;">in</span> <span style="color: #d0d0d0;">any</span>
<span style="color: #d0d0d0;">module</span> <span style="color: #d0d0d0;">directory.</span>
<span style="color: #d0d0d0;">At</span> <span style="color: #d0d0d0;">line</span><span style="background-color: #e3d2d2; color: #a61717;">:</span><span style="color: #d0d0d0;">1</span> <span style="color: #d0d0d0;">char</span><span style="background-color: #e3d2d2; color: #a61717;">:</span><span style="color: #d0d0d0;">1</span>
<span style="color: #d0d0d0;">+</span> <span style="color: #24909d;">Import-Module</span> <span style="color: #d0d0d0;">ACMEPowerShell</span>
<span style="color: #d0d0d0;">+</span> <span style="color: #d0d0d0;">~~~~~~~~~~~~~~~~~~~~~~~~~~~~</span>
<span style="color: #d0d0d0;">+</span> <span style="color: #d0d0d0;">CategoryInfo</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">ResourceUnavailable</span><span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">(ACMEPowerShell</span><span style="background-color: #e3d2d2; color: #a61717;">:</span><span style="color: #d0d0d0;">String)</span> <span style="color: #d0d0d0;">[</span><span style="color: #24909d;">Import-Module</span><span style="color: #d0d0d0;">],</span> <span style="color: #d0d0d0;">FileNotFoundException</span>
<span style="color: #d0d0d0;">+</span> <span style="color: #d0d0d0;">FullyQualifiedErrorId</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">Modules_ModuleNotFound,Microsoft.PowerShell.Commands.ImportModuleCommand</span>
<span style="color: #d0d0d0;">PS</span> <span style="color: #d0d0d0;">C</span><span style="background-color: #e3d2d2; color: #a61717;">:</span><span style="color: #d0d0d0;">\Vault></span> <span style="color: #24909d;">Import-Module</span> <span style="color: #d0d0d0;">ACMEPowerShell-IIS</span>
<span style="color: #24909d;">Import-Module</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">The</span> <span style="color: #d0d0d0;">specified</span> <span style="color: #d0d0d0;">module</span> <span style="color: #ed9d13;">'ACMEPowerShell-IIS'</span> <span style="color: #d0d0d0;">was</span> <span style="color: #d0d0d0;">not</span> <span style="color: #d0d0d0;">loaded</span> <span style="color: #d0d0d0;">because</span> <span style="color: #d0d0d0;">no</span> <span style="color: #d0d0d0;">valid</span> <span style="color: #d0d0d0;">module</span> <span style="color: #d0d0d0;">file</span> <span style="color: #d0d0d0;">was</span> <span style="color: #d0d0d0;">found</span> <span style="color: #6ab825; font-weight: bold;">in</span> <span style="color: #d0d0d0;">any</span>
<span style="color: #d0d0d0;">module</span> <span style="color: #d0d0d0;">directory.</span>
<span style="color: #d0d0d0;">At</span> <span style="color: #d0d0d0;">line</span><span style="background-color: #e3d2d2; color: #a61717;">:</span><span style="color: #d0d0d0;">1</span> <span style="color: #d0d0d0;">char</span><span style="background-color: #e3d2d2; color: #a61717;">:</span><span style="color: #d0d0d0;">1</span>
<span style="color: #d0d0d0;">+</span> <span style="color: #24909d;">Import-Module</span> <span style="color: #d0d0d0;">ACMEPowerShell-IIS</span>
<span style="color: #d0d0d0;">+</span> <span style="color: #d0d0d0;">~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~</span>
<span style="color: #d0d0d0;">+</span> <span style="color: #d0d0d0;">CategoryInfo</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">ResourceUnavailable</span><span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">(ACMEPowerShell-IIS</span><span style="background-color: #e3d2d2; color: #a61717;">:</span><span style="color: #d0d0d0;">String)</span> <span style="color: #d0d0d0;">[</span><span style="color: #24909d;">Import-Module</span><span style="color: #d0d0d0;">],</span> <span style="color: #d0d0d0;">FileNotFoundException</span>
<span style="color: #d0d0d0;">+</span> <span style="color: #d0d0d0;">FullyQualifiedErrorId</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">Modules_ModuleNotFound,Microsoft.PowerShell.Commands.ImportModuleCommand</span>
<span style="color: #d0d0d0;">PS</span> <span style="color: #d0d0d0;">C</span><span style="background-color: #e3d2d2; color: #a61717;">:</span><span style="color: #d0d0d0;">\Vault></span> <span style="color: #d0d0d0;">Install-WindowsFeature</span> <span style="color: #ed9d13;">"Web-Scripting-Tools"</span> <span style="color: #d0d0d0;">-IncludeAllSubFeature</span> <span style="color: #d0d0d0;">-IncludeManagementTools</span>
<span style="color: #d0d0d0;">Success</span> <span style="color: #d0d0d0;">Restart</span> <span style="color: #d0d0d0;">Needed</span> <span style="color: #d0d0d0;">Exit</span> <span style="color: #d0d0d0;">Code</span> <span style="color: #d0d0d0;">Feature</span> <span style="color: #d0d0d0;">Result</span>
<span style="color: #d0d0d0;">-------</span> <span style="color: #d0d0d0;">--------------</span> <span style="color: #d0d0d0;">---------</span> <span style="color: #d0d0d0;">--------------</span>
<span style="color: #d0d0d0;">True</span> <span style="color: #d0d0d0;">No</span> <span style="color: #d0d0d0;">Success</span> <span style="color: #d0d0d0;">{IIS</span> <span style="color: #d0d0d0;">Management</span> <span style="color: #d0d0d0;">Scripts</span> <span style="color: #d0d0d0;">and</span> <span style="color: #d0d0d0;">Tools}</span>
<span style="color: #d0d0d0;">WARNING</span><span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">Windows</span> <span style="color: #d0d0d0;">automatic</span> <span style="color: #d0d0d0;">updating</span> <span style="color: #d0d0d0;">is</span> <span style="color: #d0d0d0;">not</span> <span style="color: #d0d0d0;">enabled.</span> <span style="color: #d0d0d0;">To</span> <span style="color: #d0d0d0;">ensure</span> <span style="color: #d0d0d0;">that</span> <span style="color: #d0d0d0;">your</span> <span style="color: #d0d0d0;">newly-installed</span> <span style="color: #d0d0d0;">role</span> <span style="color: #d0d0d0;">or</span> <span style="color: #d0d0d0;">feature</span> <span style="color: #d0d0d0;">is</span>
<span style="color: #d0d0d0;">automatically</span> <span style="color: #d0d0d0;">updated,</span> <span style="color: #d0d0d0;">turn</span> <span style="color: #d0d0d0;">on</span> <span style="color: #d0d0d0;">Windows</span> <span style="color: #d0d0d0;">Update.</span>
<span style="color: #d0d0d0;">PS</span> <span style="color: #d0d0d0;">C</span><span style="background-color: #e3d2d2; color: #a61717;">:</span><span style="color: #d0d0d0;">\Vault></span> <span style="color: #d0d0d0;">Initialize-ACMEVault</span> <span style="color: #d0d0d0;">-BaseURI</span> <span style="color: #d0d0d0;">https</span><span style="background-color: #e3d2d2; color: #a61717;">:</span><span style="color: #d0d0d0;">//acme-staging.api.letsencrypt.org/</span>
<span style="color: #d0d0d0;">Initialize-ACMEVault</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">Vault</span> <span style="color: #d0d0d0;">root</span> <span style="color: #d0d0d0;">path</span> <span style="color: #d0d0d0;">contains</span> <span style="color: #d0d0d0;">existing</span> <span style="color: #d0d0d0;">vault</span> <span style="color: #d0d0d0;">data</span>
<span style="color: #d0d0d0;">At</span> <span style="color: #d0d0d0;">line</span><span style="background-color: #e3d2d2; color: #a61717;">:</span><span style="color: #d0d0d0;">1</span> <span style="color: #d0d0d0;">char</span><span style="background-color: #e3d2d2; color: #a61717;">:</span><span style="color: #d0d0d0;">1</span>
<span style="color: #d0d0d0;">+</span> <span style="color: #d0d0d0;">Initialize-ACMEVault</span> <span style="color: #d0d0d0;">-BaseURI</span> <span style="color: #d0d0d0;">https</span><span style="background-color: #e3d2d2; color: #a61717;">:</span><span style="color: #d0d0d0;">//acme-staging.api.letsencrypt.org/</span>
<span style="color: #d0d0d0;">+</span> <span style="color: #d0d0d0;">~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~</span>
<span style="color: #d0d0d0;">+</span> <span style="color: #d0d0d0;">CategoryInfo</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">NotSpecified</span><span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">(</span><span style="background-color: #e3d2d2; color: #a61717;">:</span><span style="color: #d0d0d0;">)</span> <span style="color: #d0d0d0;">[Initialize-ACMEVault],</span> <span style="color: #d0d0d0;">Exception</span>
<span style="color: #d0d0d0;">+</span> <span style="color: #d0d0d0;">FullyQualifiedErrorId</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">System.Exception,ACMESharp.POSH.InitializeVault</span>
<span style="color: #d0d0d0;">PS</span> <span style="color: #d0d0d0;">C</span><span style="background-color: #e3d2d2; color: #a61717;">:</span><span style="color: #d0d0d0;">\Vault></span> <span style="color: #d0d0d0;">Initialize-ACMEVault</span>
<span style="color: #d0d0d0;">Initialize-ACMEVault</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">Vault</span> <span style="color: #d0d0d0;">root</span> <span style="color: #d0d0d0;">path</span> <span style="color: #d0d0d0;">contains</span> <span style="color: #d0d0d0;">existing</span> <span style="color: #d0d0d0;">vault</span> <span style="color: #d0d0d0;">data</span>
<span style="color: #d0d0d0;">At</span> <span style="color: #d0d0d0;">line</span><span style="background-color: #e3d2d2; color: #a61717;">:</span><span style="color: #d0d0d0;">1</span> <span style="color: #d0d0d0;">char</span><span style="background-color: #e3d2d2; color: #a61717;">:</span><span style="color: #d0d0d0;">1</span>
<span style="color: #d0d0d0;">+</span> <span style="color: #d0d0d0;">Initialize-ACMEVault</span>
<span style="color: #d0d0d0;">+</span> <span style="color: #d0d0d0;">~~~~~~~~~~~~~~~~~~~~</span>
<span style="color: #d0d0d0;">+</span> <span style="color: #d0d0d0;">CategoryInfo</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">NotSpecified</span><span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">(</span><span style="background-color: #e3d2d2; color: #a61717;">:</span><span style="color: #d0d0d0;">)</span> <span style="color: #d0d0d0;">[Initialize-ACMEVault],</span> <span style="color: #d0d0d0;">Exception</span>
<span style="color: #d0d0d0;">+</span> <span style="color: #d0d0d0;">FullyQualifiedErrorId</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">System.Exception,ACMESharp.POSH.InitializeVault</span>
<span style="color: #d0d0d0;">PS</span> <span style="color: #d0d0d0;">C</span><span style="background-color: #e3d2d2; color: #a61717;">:</span><span style="color: #d0d0d0;">\Vault></span> <span style="color: #d0d0d0;">Initialize-ACMEVault</span>
<span style="color: #d0d0d0;">Initialize-ACMEVault</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">Vault</span> <span style="color: #d0d0d0;">root</span> <span style="color: #d0d0d0;">path</span> <span style="color: #d0d0d0;">contains</span> <span style="color: #d0d0d0;">existing</span> <span style="color: #d0d0d0;">vault</span> <span style="color: #d0d0d0;">data</span>
<span style="color: #d0d0d0;">At</span> <span style="color: #d0d0d0;">line</span><span style="background-color: #e3d2d2; color: #a61717;">:</span><span style="color: #d0d0d0;">1</span> <span style="color: #d0d0d0;">char</span><span style="background-color: #e3d2d2; color: #a61717;">:</span><span style="color: #d0d0d0;">1</span>
<span style="color: #d0d0d0;">+</span> <span style="color: #d0d0d0;">Initialize-ACMEVault</span>
<span style="color: #d0d0d0;">+</span> <span style="color: #d0d0d0;">~~~~~~~~~~~~~~~~~~~~</span>
<span style="color: #d0d0d0;">+</span> <span style="color: #d0d0d0;">CategoryInfo</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">NotSpecified</span><span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">(</span><span style="background-color: #e3d2d2; color: #a61717;">:</span><span style="color: #d0d0d0;">)</span> <span style="color: #d0d0d0;">[Initialize-ACMEVault],</span> <span style="color: #d0d0d0;">Exception</span>
<span style="color: #d0d0d0;">+</span> <span style="color: #d0d0d0;">FullyQualifiedErrorId</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">System.Exception,ACMESharp.POSH.InitializeVault</span>
<span style="color: #d0d0d0;">PS</span> <span style="color: #d0d0d0;">C</span><span style="background-color: #e3d2d2; color: #a61717;">:</span><span style="color: #d0d0d0;">\Vault></span> <span style="color: #24909d;">New-ACMERegistration</span> <span style="color: #d0d0d0;">-Contacts</span> <span style="color: #d0d0d0;">mailto</span><span style="background-color: #e3d2d2; color: #a61717;">:</span><span style="color: #d0d0d0;">nutthaphon</span><span style="color: #40ffff;">@gmail</span><span style="color: #d0d0d0;">.com</span> <span style="color: #d0d0d0;">-AcceptTos</span>
<span style="color: #d0d0d0;">Contacts</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">{mailto</span><span style="background-color: #e3d2d2; color: #a61717;">:</span><span style="color: #d0d0d0;">nutthaphon</span><span style="color: #40ffff;">@gmail</span><span style="color: #d0d0d0;">.com}</span>
<span style="color: #d0d0d0;">PublicKey</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">{</span> <span style="color: #d0d0d0;">e</span> <span style="color: #d0d0d0;">=</span> <span style="color: #d0d0d0;">AQAB,</span> <span style="color: #d0d0d0;">kty</span> <span style="color: #d0d0d0;">=</span> <span style="color: #d0d0d0;">RSA,</span> <span style="color: #d0d0d0;">n</span> <span style="color: #d0d0d0;">=</span> <span style="color: #d0d0d0;">v9IP2ZkBxDiMmfLkI7qLbloslZDmEMa7JsOKr76ckbaDlXAo5hkt1ddPwJjcU2k-LS6EZyNG</span>
<span style="color: #d0d0d0;">ewBwcNbq2F5ILrlYi0PA4h-icWivyjyJeJ-Fi2p2DqlfPVMfMdUVZ-ZZgxMMSj0uoUZvrjWUl16kPVUAoFe-aY9mvXXP3SndFQY</span>
<span style="color: #d0d0d0;">j6t4R0Al5AVTFuZw2aYV5rGvTTQRG_cTOrw3UBiMvrSZ_16td1zZCkzuH21E4mNDMS2-cufn34Jc9RCGjItuack81PsaHbUJUbz</span>
<span style="color: #d0d0d0;">wXv6KioDj--0xz39s6aX_DWqc9lfDeEaQh-_ZEU7zEZyqF9GarUg9eNHoV2ByOAPh92SfGBQ</span> <span style="color: #d0d0d0;">}</span>
<span style="color: #d0d0d0;">RecoveryKey</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span>
<span style="color: #d0d0d0;">RegistrationUri</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">https</span><span style="background-color: #e3d2d2; color: #a61717;">:</span><span style="color: #d0d0d0;">//acme-v01.api.letsencrypt.org/acme/reg/10801071</span>
<span style="color: #d0d0d0;">Links</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">{<https</span><span style="background-color: #e3d2d2; color: #a61717;">:</span><span style="color: #d0d0d0;">//acme-v01.api.letsencrypt.org/acme/</span><span style="color: #24909d;">new-authz</span><span style="color: #d0d0d0;">></span><span style="background-color: #e3d2d2; color: #a61717;">;</span><span style="color: #d0d0d0;">rel=</span><span style="color: #ed9d13;">"next"</span><span style="color: #d0d0d0;">,</span>
<span style="color: #d0d0d0;"><https</span><span style="background-color: #e3d2d2; color: #a61717;">:</span><span style="color: #d0d0d0;">//letsencrypt.org/documents/LE-SA-v1.1.1-August-1-2016.pdf></span><span style="background-color: #e3d2d2; color: #a61717;">;</span><span style="color: #d0d0d0;">rel=</span><span style="color: #ed9d13;">"terms-of-service"</span><span style="color: #d0d0d0;">}</span>
<span style="color: #d0d0d0;">TosLinkUri</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">https</span><span style="background-color: #e3d2d2; color: #a61717;">:</span><span style="color: #d0d0d0;">//letsencrypt.org/documents/LE-SA-v1.1.1-August-1-2016.pdf</span>
<span style="color: #d0d0d0;">TosAgreementUri</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">https</span><span style="background-color: #e3d2d2; color: #a61717;">:</span><span style="color: #d0d0d0;">//letsencrypt.org/documents/LE-SA-v1.1.1-August-1-2016.pdf</span>
<span style="color: #d0d0d0;">AuthorizationsUri</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span>
<span style="color: #d0d0d0;">CertificatesUri</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span>
<span style="color: #d0d0d0;">PS</span> <span style="color: #d0d0d0;">C</span><span style="background-color: #e3d2d2; color: #a61717;">:</span><span style="color: #d0d0d0;">\Vault></span> <span style="color: #24909d;">New-ACMEIdentifier</span> <span style="color: #d0d0d0;">-Dns</span> <span style="color: #d0d0d0;">mserver1.dataascii.com</span> <span style="color: #d0d0d0;">-Alias</span> <span style="color: #d0d0d0;">dns1</span>
<span style="color: #24909d;">New-ACMEIdentifier</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">An</span> <span style="color: #d0d0d0;">item</span> <span style="color: #d0d0d0;">with</span> <span style="color: #d0d0d0;">the</span> <span style="color: #d0d0d0;">same</span> <span style="color: #d0d0d0;">key</span> <span style="color: #d0d0d0;">has</span> <span style="color: #d0d0d0;">already</span> <span style="color: #d0d0d0;">been</span> <span style="color: #d0d0d0;">added.</span>
<span style="color: #d0d0d0;">At</span> <span style="color: #d0d0d0;">line</span><span style="background-color: #e3d2d2; color: #a61717;">:</span><span style="color: #d0d0d0;">1</span> <span style="color: #d0d0d0;">char</span><span style="background-color: #e3d2d2; color: #a61717;">:</span><span style="color: #d0d0d0;">1</span>
<span style="color: #d0d0d0;">+</span> <span style="color: #24909d;">New-ACMEIdentifier</span> <span style="color: #d0d0d0;">-Dns</span> <span style="color: #d0d0d0;">mserver1.dataascii.com</span> <span style="color: #d0d0d0;">-Alias</span> <span style="color: #d0d0d0;">dns1</span>
<span style="color: #d0d0d0;">+</span> <span style="color: #d0d0d0;">~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~</span>
<span style="color: #d0d0d0;">+</span> <span style="color: #d0d0d0;">CategoryInfo</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">NotSpecified</span><span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">(</span><span style="background-color: #e3d2d2; color: #a61717;">:</span><span style="color: #d0d0d0;">)</span> <span style="color: #d0d0d0;">[</span><span style="color: #24909d;">New-ACMEIdentifier</span><span style="color: #d0d0d0;">],</span> <span style="color: #d0d0d0;">ArgumentException</span>
<span style="color: #d0d0d0;">+</span> <span style="color: #d0d0d0;">FullyQualifiedErrorId</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">System.ArgumentException,ACMESharp.POSH.NewIdentifier</span>
<span style="color: #d0d0d0;">PS</span> <span style="color: #d0d0d0;">C</span><span style="background-color: #e3d2d2; color: #a61717;">:</span><span style="color: #d0d0d0;">\Vault></span> <span style="color: #24909d;">New-ACMEIdentifier</span> <span style="color: #d0d0d0;">-Dns</span> <span style="color: #d0d0d0;">mserver1.dataascii.com</span> <span style="color: #d0d0d0;">-Alias</span> <span style="color: #d0d0d0;">mserver1</span>
<span style="color: #d0d0d0;">IdentifierPart</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">ACMESharp.Messages.IdentifierPart</span>
<span style="color: #d0d0d0;">IdentifierType</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">dns</span>
<span style="color: #d0d0d0;">Identifier</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">mserver1.dataascii.com</span>
<span style="color: #d0d0d0;">Uri</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">https</span><span style="background-color: #e3d2d2; color: #a61717;">:</span><span style="color: #d0d0d0;">//acme-v01.api.letsencrypt.org/acme/authz/pfSORbGt_uujkL5hlZvxFGcO4R5mo_x2tmn9qKcJCA8</span>
<span style="color: #d0d0d0;">Status</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">pending</span>
<span style="color: #d0d0d0;">Expires</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">3/20/2017</span> <span style="color: #d0d0d0;">2</span><span style="background-color: #e3d2d2; color: #a61717;">:</span><span style="color: #d0d0d0;">43</span><span style="background-color: #e3d2d2; color: #a61717;">:</span><span style="color: #d0d0d0;">13</span> <span style="color: #d0d0d0;">PM</span>
<span style="color: #d0d0d0;">Challenges</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">{,</span> <span style="color: #d0d0d0;">,</span> <span style="color: #d0d0d0;">}</span>
<span style="color: #d0d0d0;">Combinations</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">{1,</span> <span style="color: #d0d0d0;">0,</span> <span style="color: #d0d0d0;">2}</span>
<span style="color: #d0d0d0;">PS</span> <span style="color: #d0d0d0;">C</span><span style="background-color: #e3d2d2; color: #a61717;">:</span><span style="color: #d0d0d0;">\Vault></span> <span style="color: #24909d;">Complete-ACMEChallenge</span> <span style="color: #d0d0d0;">mserver1</span> <span style="color: #d0d0d0;">-ChallengeType</span> <span style="color: #d0d0d0;">http-01</span> <span style="color: #d0d0d0;">-Handler</span> <span style="color: #d0d0d0;">iis</span> <span style="color: #d0d0d0;">-HandlerParameters</span> <span style="background-color: #e3d2d2; color: #a61717;">@</span><span style="color: #d0d0d0;">{</span> <span style="color: #d0d0d0;">WebSiteRef</span> <span style="color: #d0d0d0;">=</span> <span style="color: #ed9d13;">'Def</span>
<span style="color: #ed9d13;">ault Web Site'</span> <span style="color: #d0d0d0;">}</span>
<span style="color: #d0d0d0;">IdentifierPart</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">ACMESharp.Messages.IdentifierPart</span>
<span style="color: #d0d0d0;">IdentifierType</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">dns</span>
<span style="color: #d0d0d0;">Identifier</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">mserver1.dataascii.com</span>
<span style="color: #d0d0d0;">Uri</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">https</span><span style="background-color: #e3d2d2; color: #a61717;">:</span><span style="color: #d0d0d0;">//acme-v01.api.letsencrypt.org/acme/authz/pfSORbGt_uujkL5hlZvxFGcO4R5mo_x2tmn9qKcJCA8</span>
<span style="color: #d0d0d0;">Status</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">pending</span>
<span style="color: #d0d0d0;">Expires</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">3/20/2017</span> <span style="color: #d0d0d0;">2</span><span style="background-color: #e3d2d2; color: #a61717;">:</span><span style="color: #d0d0d0;">43</span><span style="background-color: #e3d2d2; color: #a61717;">:</span><span style="color: #d0d0d0;">13</span> <span style="color: #d0d0d0;">PM</span>
<span style="color: #d0d0d0;">Challenges</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">{,</span> <span style="color: #d0d0d0;">,</span> <span style="color: #d0d0d0;">iis}</span>
<span style="color: #d0d0d0;">Combinations</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">{1,</span> <span style="color: #d0d0d0;">0,</span> <span style="color: #d0d0d0;">2}</span>
<span style="color: #d0d0d0;">PS</span> <span style="color: #d0d0d0;">C</span><span style="background-color: #e3d2d2; color: #a61717;">:</span><span style="color: #d0d0d0;">\Vault></span> <span style="color: #d0d0d0;">Submit-ACMEChallenge</span> <span style="color: #d0d0d0;">mserver1</span> <span style="color: #d0d0d0;">-ChallengeType</span> <span style="color: #d0d0d0;">http-01</span>
<span style="color: #d0d0d0;">IdentifierPart</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">ACMESharp.Messages.IdentifierPart</span>
<span style="color: #d0d0d0;">IdentifierType</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">dns</span>
<span style="color: #d0d0d0;">Identifier</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">mserver1.dataascii.com</span>
<span style="color: #d0d0d0;">Uri</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">https</span><span style="background-color: #e3d2d2; color: #a61717;">:</span><span style="color: #d0d0d0;">//acme-v01.api.letsencrypt.org/acme/authz/pfSORbGt_uujkL5hlZvxFGcO4R5mo_x2tmn9qKcJCA8</span>
<span style="color: #d0d0d0;">Status</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">pending</span>
<span style="color: #d0d0d0;">Expires</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">3/20/2017</span> <span style="color: #d0d0d0;">2</span><span style="background-color: #e3d2d2; color: #a61717;">:</span><span style="color: #d0d0d0;">43</span><span style="background-color: #e3d2d2; color: #a61717;">:</span><span style="color: #d0d0d0;">13</span> <span style="color: #d0d0d0;">PM</span>
<span style="color: #d0d0d0;">Challenges</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">{,</span> <span style="color: #d0d0d0;">,</span> <span style="color: #d0d0d0;">iis}</span>
<span style="color: #d0d0d0;">Combinations</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">{1,</span> <span style="color: #d0d0d0;">0,</span> <span style="color: #d0d0d0;">2}</span>
<span style="color: #d0d0d0;">PS</span> <span style="color: #d0d0d0;">C</span><span style="background-color: #e3d2d2; color: #a61717;">:</span><span style="color: #d0d0d0;">\Vault></span> <span style="color: #d0d0d0;">(</span><span style="color: #24909d;">Update-ACMEIdentifier</span> <span style="color: #d0d0d0;">mserver1</span> <span style="color: #d0d0d0;">-ChallengeType</span> <span style="color: #d0d0d0;">http-01).Challenges</span> <span style="color: #d0d0d0;">|</span> <span style="color: #24909d;">Where-Object</span> <span style="color: #d0d0d0;">{</span><span style="color: #40ffff;">$_</span><span style="color: #d0d0d0;">.Type</span> <span style="color: #d0d0d0;">-eq</span> <span style="color: #ed9d13;">"http-01"</span><span style="color: #d0d0d0;">}</span>
<span style="color: #d0d0d0;">ChallengePart</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">ACMESharp.Messages.ChallengePart</span>
<span style="color: #d0d0d0;">Challenge</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">ACMESharp.ACME.HttpChallenge</span>
<span style="color: #d0d0d0;">Type</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">http-01</span>
<span style="color: #d0d0d0;">Uri</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">https</span><span style="background-color: #e3d2d2; color: #a61717;">:</span><span style="color: #d0d0d0;">//acme-v01.api.letsencrypt.org/acme/challenge/pfSORbGt_uujkL5hlZvxFGcO4R5mo_x2tmn9qKcJCA</span>
<span style="color: #d0d0d0;">8/792581011</span>
<span style="color: #d0d0d0;">Token</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">EQGsWNjNm0Jkcc9-dqetE7cchCzvaY5J6I9_B7DC7Fw</span>
<span style="color: #d0d0d0;">Status</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">invalid</span>
<span style="color: #d0d0d0;">OldChallengeAnswer</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">[,</span> <span style="color: #d0d0d0;">]</span>
<span style="color: #d0d0d0;">ChallengeAnswerMessage</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span>
<span style="color: #d0d0d0;">HandlerName</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">iis</span>
<span style="color: #d0d0d0;">HandlerHandleDate</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">3/13/2017</span> <span style="color: #d0d0d0;">9</span><span style="background-color: #e3d2d2; color: #a61717;">:</span><span style="color: #d0d0d0;">44</span><span style="background-color: #e3d2d2; color: #a61717;">:</span><span style="color: #d0d0d0;">12</span> <span style="color: #d0d0d0;">PM</span>
<span style="color: #d0d0d0;">HandlerCleanUpDate</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span>
<span style="color: #d0d0d0;">SubmitDate</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">3/13/2017</span> <span style="color: #d0d0d0;">9</span><span style="background-color: #e3d2d2; color: #a61717;">:</span><span style="color: #d0d0d0;">45</span><span style="background-color: #e3d2d2; color: #a61717;">:</span><span style="color: #d0d0d0;">42</span> <span style="color: #d0d0d0;">PM</span>
<span style="color: #d0d0d0;">SubmitResponse</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">{StatusCode,</span> <span style="color: #d0d0d0;">Headers,</span> <span style="color: #d0d0d0;">Links,</span> <span style="color: #d0d0d0;">RawContent...}</span>
<span style="color: #d0d0d0;">PS</span> <span style="color: #d0d0d0;">C</span><span style="background-color: #e3d2d2; color: #a61717;">:</span><span style="color: #d0d0d0;">\Vault></span> <span style="color: #24909d;">New-ACMEIdentifier</span> <span style="color: #d0d0d0;">-Dns</span> <span style="color: #d0d0d0;">mserver1.dataascii.com</span> <span style="color: #d0d0d0;">-Alias</span> <span style="color: #d0d0d0;">mserver1</span>
<span style="color: #24909d;">New-ACMEIdentifier</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">An</span> <span style="color: #d0d0d0;">item</span> <span style="color: #d0d0d0;">with</span> <span style="color: #d0d0d0;">the</span> <span style="color: #d0d0d0;">same</span> <span style="color: #d0d0d0;">key</span> <span style="color: #d0d0d0;">has</span> <span style="color: #d0d0d0;">already</span> <span style="color: #d0d0d0;">been</span> <span style="color: #d0d0d0;">added.</span>
<span style="color: #d0d0d0;">At</span> <span style="color: #d0d0d0;">line</span><span style="background-color: #e3d2d2; color: #a61717;">:</span><span style="color: #d0d0d0;">1</span> <span style="color: #d0d0d0;">char</span><span style="background-color: #e3d2d2; color: #a61717;">:</span><span style="color: #d0d0d0;">1</span>
<span style="color: #d0d0d0;">+</span> <span style="color: #24909d;">New-ACMEIdentifier</span> <span style="color: #d0d0d0;">-Dns</span> <span style="color: #d0d0d0;">mserver1.dataascii.com</span> <span style="color: #d0d0d0;">-Alias</span> <span style="color: #d0d0d0;">mserver1</span>
<span style="color: #d0d0d0;">+</span> <span style="color: #d0d0d0;">~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~</span>
<span style="color: #d0d0d0;">+</span> <span style="color: #d0d0d0;">CategoryInfo</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">NotSpecified</span><span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">(</span><span style="background-color: #e3d2d2; color: #a61717;">:</span><span style="color: #d0d0d0;">)</span> <span style="color: #d0d0d0;">[</span><span style="color: #24909d;">New-ACMEIdentifier</span><span style="color: #d0d0d0;">],</span> <span style="color: #d0d0d0;">ArgumentException</span>
<span style="color: #d0d0d0;">+</span> <span style="color: #d0d0d0;">FullyQualifiedErrorId</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">System.ArgumentException,ACMESharp.POSH.NewIdentifier</span>
<span style="color: #d0d0d0;">PS</span> <span style="color: #d0d0d0;">C</span><span style="background-color: #e3d2d2; color: #a61717;">:</span><span style="color: #d0d0d0;">\Vault></span> <span style="color: #24909d;">New-ACMEIdentifier</span> <span style="color: #d0d0d0;">-Dns</span> <span style="color: #d0d0d0;">mserver1.dataascii.com</span> <span style="color: #d0d0d0;">-Alias</span> <span style="color: #d0d0d0;">mserver1</span>
<span style="color: #d0d0d0;">PS</span> <span style="color: #d0d0d0;">C</span><span style="background-color: #e3d2d2; color: #a61717;">:</span><span style="color: #d0d0d0;">\Vault></span> <span style="color: #24909d;">New-ACMEIdentifier</span> <span style="color: #d0d0d0;">-Dns</span> <span style="color: #d0d0d0;">mserver1.dataascii.com</span> <span style="color: #d0d0d0;">-Alias</span> <span style="color: #d0d0d0;">mserver2</span>
<span style="color: #d0d0d0;">IdentifierPart</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">ACMESharp.Messages.IdentifierPart</span>
<span style="color: #d0d0d0;">IdentifierType</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">dns</span>
<span style="color: #d0d0d0;">Identifier</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">mserver1.dataascii.com</span>
<span style="color: #d0d0d0;">Uri</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">https</span><span style="background-color: #e3d2d2; color: #a61717;">:</span><span style="color: #d0d0d0;">//acme-v01.api.letsencrypt.org/acme/authz/XMBoNhTkW5363As5EcTT4D3ONE0DkRwwWBSVN-4mfZs</span>
<span style="color: #d0d0d0;">Status</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">pending</span>
<span style="color: #d0d0d0;">Expires</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">3/20/2017</span> <span style="color: #d0d0d0;">2</span><span style="background-color: #e3d2d2; color: #a61717;">:</span><span style="color: #d0d0d0;">49</span><span style="background-color: #e3d2d2; color: #a61717;">:</span><span style="color: #d0d0d0;">00</span> <span style="color: #d0d0d0;">PM</span>
<span style="color: #d0d0d0;">Challenges</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">{,</span> <span style="color: #d0d0d0;">,</span> <span style="color: #d0d0d0;">}</span>
<span style="color: #d0d0d0;">Combinations</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">{1,</span> <span style="color: #d0d0d0;">0,</span> <span style="color: #d0d0d0;">2}</span>
<span style="color: #d0d0d0;">PS</span> <span style="color: #d0d0d0;">C</span><span style="background-color: #e3d2d2; color: #a61717;">:</span><span style="color: #d0d0d0;">\Vault></span> <span style="color: #24909d;">Complete-ACMEChallenge</span> <span style="color: #d0d0d0;">dns1</span> <span style="color: #d0d0d0;">-ChallengeType</span> <span style="color: #d0d0d0;">http-01</span> <span style="color: #d0d0d0;">-Handler</span> <span style="color: #d0d0d0;">manual</span>
<span style="color: #d0d0d0;">IdentifierPart</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">ACMESharp.Messages.IdentifierPart</span>
<span style="color: #d0d0d0;">IdentifierType</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">dns</span>
<span style="color: #d0d0d0;">Identifier</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">nutthaphon.thddns.net</span>
<span style="color: #d0d0d0;">Uri</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">https</span><span style="background-color: #e3d2d2; color: #a61717;">:</span><span style="color: #d0d0d0;">//acme-v01.api.letsencrypt.org/acme/authz/issiGydOyE8wNHOAa2pCwYCGNGhbt7RiBwTfttrQLdk</span>
<span style="color: #d0d0d0;">Status</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">pending</span>
<span style="color: #d0d0d0;">Expires</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">1/31/2017</span> <span style="color: #d0d0d0;">8</span><span style="background-color: #e3d2d2; color: #a61717;">:</span><span style="color: #d0d0d0;">12</span><span style="background-color: #e3d2d2; color: #a61717;">:</span><span style="color: #d0d0d0;">51</span> <span style="color: #d0d0d0;">AM</span>
<span style="color: #d0d0d0;">Challenges</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">{,</span> <span style="color: #d0d0d0;">manual,</span> <span style="color: #d0d0d0;">}</span>
<span style="color: #d0d0d0;">Combinations</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">{2,</span> <span style="color: #d0d0d0;">0,</span> <span style="color: #d0d0d0;">1}</span>
<span style="color: #d0d0d0;">PS</span> <span style="color: #d0d0d0;">C</span><span style="background-color: #e3d2d2; color: #a61717;">:</span><span style="color: #d0d0d0;">\Vault></span> <span style="color: #d0d0d0;">Submit-ACMEChallenge</span> <span style="color: #d0d0d0;">mserver2</span> <span style="color: #d0d0d0;">-ChallengeType</span> <span style="color: #d0d0d0;">http-01</span>
<span style="color: #d0d0d0;">Submit-ACMEChallenge</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">challenge</span> <span style="color: #d0d0d0;">has</span> <span style="color: #d0d0d0;">not</span> <span style="color: #d0d0d0;">been</span> <span style="color: #d0d0d0;">decoded</span>
<span style="color: #d0d0d0;">At</span> <span style="color: #d0d0d0;">line</span><span style="background-color: #e3d2d2; color: #a61717;">:</span><span style="color: #d0d0d0;">1</span> <span style="color: #d0d0d0;">char</span><span style="background-color: #e3d2d2; color: #a61717;">:</span><span style="color: #d0d0d0;">1</span>
<span style="color: #d0d0d0;">+</span> <span style="color: #d0d0d0;">Submit-ACMEChallenge</span> <span style="color: #d0d0d0;">mserver2</span> <span style="color: #d0d0d0;">-ChallengeType</span> <span style="color: #d0d0d0;">http-01</span>
<span style="color: #d0d0d0;">+</span> <span style="color: #d0d0d0;">~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~</span>
<span style="color: #d0d0d0;">+</span> <span style="color: #d0d0d0;">CategoryInfo</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">NotSpecified</span><span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">(</span><span style="background-color: #e3d2d2; color: #a61717;">:</span><span style="color: #d0d0d0;">)</span> <span style="color: #d0d0d0;">[Submit-ACMEChallenge],</span> <span style="color: #d0d0d0;">InvalidOperationException</span>
<span style="color: #d0d0d0;">+</span> <span style="color: #d0d0d0;">FullyQualifiedErrorId</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">System.InvalidOperationException,ACMESharp.POSH.SubmitChallenge</span>
<span style="color: #d0d0d0;">PS</span> <span style="color: #d0d0d0;">C</span><span style="background-color: #e3d2d2; color: #a61717;">:</span><span style="color: #d0d0d0;">\Vault></span> <span style="color: #24909d;">New-ACMEIdentifier</span> <span style="color: #d0d0d0;">-Dns</span> <span style="color: #d0d0d0;">mserver1.dataascii.com</span> <span style="color: #d0d0d0;">-Alias</span> <span style="color: #d0d0d0;">mserver3</span>
<span style="color: #d0d0d0;">IdentifierPart</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">ACMESharp.Messages.IdentifierPart</span>
<span style="color: #d0d0d0;">IdentifierType</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">dns</span>
<span style="color: #d0d0d0;">Identifier</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">mserver1.dataascii.com</span>
<span style="color: #d0d0d0;">Uri</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">https</span><span style="background-color: #e3d2d2; color: #a61717;">:</span><span style="color: #d0d0d0;">//acme-v01.api.letsencrypt.org/acme/authz/KoKUWptBR61KJWILZObPmZLxsK_nDhhQvKQRYrqMg5w</span>
<span style="color: #d0d0d0;">Status</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">pending</span>
<span style="color: #d0d0d0;">Expires</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">3/20/2017</span> <span style="color: #d0d0d0;">2</span><span style="background-color: #e3d2d2; color: #a61717;">:</span><span style="color: #d0d0d0;">51</span><span style="background-color: #e3d2d2; color: #a61717;">:</span><span style="color: #d0d0d0;">16</span> <span style="color: #d0d0d0;">PM</span>
<span style="color: #d0d0d0;">Challenges</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">{,</span> <span style="color: #d0d0d0;">,</span> <span style="color: #d0d0d0;">}</span>
<span style="color: #d0d0d0;">Combinations</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">{2,</span> <span style="color: #d0d0d0;">0,</span> <span style="color: #d0d0d0;">1}</span>
<span style="color: #d0d0d0;">PS</span> <span style="color: #d0d0d0;">C</span><span style="background-color: #e3d2d2; color: #a61717;">:</span><span style="color: #d0d0d0;">\Vault></span> <span style="color: #24909d;">Complete-ACMEChallenge</span> <span style="color: #d0d0d0;">mserver3</span> <span style="color: #d0d0d0;">-ChallengeType</span> <span style="color: #d0d0d0;">dns-01</span> <span style="color: #d0d0d0;">-Handler</span> <span style="color: #d0d0d0;">manual</span>
<span style="color: #d0d0d0;">==</span> <span style="color: #d0d0d0;">Manual</span> <span style="color: #d0d0d0;">Challenge</span> <span style="color: #d0d0d0;">Handler</span> <span style="color: #d0d0d0;">-</span> <span style="color: #d0d0d0;">DNS</span> <span style="color: #d0d0d0;">==</span>
<span style="color: #d0d0d0;">*</span> <span style="color: #d0d0d0;">Handle</span> <span style="color: #d0d0d0;">Time</span><span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">[3/13/2017</span> <span style="color: #d0d0d0;">9</span><span style="background-color: #e3d2d2; color: #a61717;">:</span><span style="color: #d0d0d0;">51</span><span style="background-color: #e3d2d2; color: #a61717;">:</span><span style="color: #d0d0d0;">49</span> <span style="color: #d0d0d0;">PM]</span>
<span style="color: #d0d0d0;">*</span> <span style="color: #d0d0d0;">Challenge</span> <span style="color: #d0d0d0;">Token</span><span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #40ffff;">[PTtQQCNHSfxePANrnalb4uMqayYFQx3RUftr19iHkT4]</span>
<span style="color: #d0d0d0;">To</span> <span style="color: #d0d0d0;">complete</span> <span style="color: #d0d0d0;">this</span> <span style="color: #d0d0d0;">Challenge</span> <span style="color: #d0d0d0;">please</span> <span style="color: #d0d0d0;">create</span> <span style="color: #d0d0d0;">a</span> <span style="color: #d0d0d0;">new</span> <span style="color: #d0d0d0;">Resource</span>
<span style="color: #d0d0d0;">Record</span> <span style="color: #d0d0d0;">(RR)</span> <span style="color: #d0d0d0;">with</span> <span style="color: #d0d0d0;">the</span> <span style="color: #d0d0d0;">following</span> <span style="color: #d0d0d0;">characteristics</span><span style="background-color: #e3d2d2; color: #a61717;">:</span>
<span style="color: #d0d0d0;">*</span> <span style="color: #d0d0d0;">RR</span> <span style="color: #d0d0d0;">Type</span><span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #40ffff;">[TXT]</span>
<span style="color: #d0d0d0;">*</span> <span style="color: #d0d0d0;">RR</span> <span style="color: #d0d0d0;">Name</span><span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">[_acme-challenge.mserver1.dataascii.com]</span>
<span style="color: #d0d0d0;">*</span> <span style="color: #d0d0d0;">RR</span> <span style="color: #d0d0d0;">Value</span><span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">[3x12NeWyGjqZMLOZBMkOv1a39PvhyKbO1O8twroe8qA]</span>
<span style="color: #d0d0d0;">------------------------------------</span>
<span style="color: #d0d0d0;">IdentifierPart</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">ACMESharp.Messages.IdentifierPart</span>
<span style="color: #d0d0d0;">IdentifierType</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">dns</span>
<span style="color: #d0d0d0;">Identifier</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">mserver1.dataascii.com</span>
<span style="color: #d0d0d0;">Uri</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">https</span><span style="background-color: #e3d2d2; color: #a61717;">:</span><span style="color: #d0d0d0;">//acme-v01.api.letsencrypt.org/acme/authz/KoKUWptBR61KJWILZObPmZLxsK_nDhhQvKQRYrqMg5w</span>
<span style="color: #d0d0d0;">Status</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">pending</span>
<span style="color: #d0d0d0;">Expires</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">3/20/2017</span> <span style="color: #d0d0d0;">2</span><span style="background-color: #e3d2d2; color: #a61717;">:</span><span style="color: #d0d0d0;">51</span><span style="background-color: #e3d2d2; color: #a61717;">:</span><span style="color: #d0d0d0;">16</span> <span style="color: #d0d0d0;">PM</span>
<span style="color: #d0d0d0;">Challenges</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">{,</span> <span style="color: #d0d0d0;">,</span> <span style="color: #d0d0d0;">manual}</span>
<span style="color: #d0d0d0;">Combinations</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">{2,</span> <span style="color: #d0d0d0;">0,</span> <span style="color: #d0d0d0;">1}</span>
<span style="color: #d0d0d0;">PS</span> <span style="color: #d0d0d0;">C</span><span style="background-color: #e3d2d2; color: #a61717;">:</span><span style="color: #d0d0d0;">\Vault></span> <span style="color: #d0d0d0;">Submit-ACMEChallenge</span> <span style="color: #d0d0d0;">mserver3</span> <span style="color: #d0d0d0;">-ChallengeType</span> <span style="color: #d0d0d0;">dns-01</span>
<span style="color: #d0d0d0;">IdentifierPart</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">ACMESharp.Messages.IdentifierPart</span>
<span style="color: #d0d0d0;">IdentifierType</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">dns</span>
<span style="color: #d0d0d0;">Identifier</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">mserver1.dataascii.com</span>
<span style="color: #d0d0d0;">Uri</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">https</span><span style="background-color: #e3d2d2; color: #a61717;">:</span><span style="color: #d0d0d0;">//acme-v01.api.letsencrypt.org/acme/authz/KoKUWptBR61KJWILZObPmZLxsK_nDhhQvKQRYrqMg5w</span>
<span style="color: #d0d0d0;">Status</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">pending</span>
<span style="color: #d0d0d0;">Expires</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">3/20/2017</span> <span style="color: #d0d0d0;">2</span><span style="background-color: #e3d2d2; color: #a61717;">:</span><span style="color: #d0d0d0;">51</span><span style="background-color: #e3d2d2; color: #a61717;">:</span><span style="color: #d0d0d0;">16</span> <span style="color: #d0d0d0;">PM</span>
<span style="color: #d0d0d0;">Challenges</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">{,</span> <span style="color: #d0d0d0;">,</span> <span style="color: #d0d0d0;">manual}</span>
<span style="color: #d0d0d0;">Combinations</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">{2,</span> <span style="color: #d0d0d0;">0,</span> <span style="color: #d0d0d0;">1}</span>
<span style="color: #d0d0d0;">PS</span> <span style="color: #d0d0d0;">C</span><span style="background-color: #e3d2d2; color: #a61717;">:</span><span style="color: #d0d0d0;">\Vault></span> <span style="color: #d0d0d0;">(</span><span style="color: #24909d;">Update-ACMEIdentifier</span> <span style="color: #d0d0d0;">mserver3</span> <span style="color: #d0d0d0;">-ChallengeType</span> <span style="color: #d0d0d0;">dns-01).Challenges</span> <span style="color: #d0d0d0;">|</span> <span style="color: #24909d;">Where-Object</span> <span style="color: #d0d0d0;">{</span><span style="color: #40ffff;">$_</span><span style="color: #d0d0d0;">.Type</span> <span style="color: #d0d0d0;">-eq</span> <span style="color: #ed9d13;">"dns-01"</span><span style="color: #d0d0d0;">}</span>
<span style="color: #d0d0d0;">ChallengePart</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">ACMESharp.Messages.ChallengePart</span>
<span style="color: #d0d0d0;">Challenge</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">ACMESharp.ACME.DnsChallenge</span>
<span style="color: #d0d0d0;">Type</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">dns-01</span>
<span style="color: #d0d0d0;">Uri</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">https</span><span style="background-color: #e3d2d2; color: #a61717;">:</span><span style="color: #d0d0d0;">//acme-v01.api.letsencrypt.org/acme/challenge/KoKUWptBR61KJWILZObPmZLxsK_nDhhQvKQRYrqMg5</span>
<span style="color: #d0d0d0;">w/792616897</span>
<span style="color: #d0d0d0;">Token</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">PTtQQCNHSfxePANrnalb4uMqayYFQx3RUftr19iHkT4</span>
<span style="color: #d0d0d0;">Status</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">valid</span>
<span style="color: #d0d0d0;">OldChallengeAnswer</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">[,</span> <span style="color: #d0d0d0;">]</span>
<span style="color: #d0d0d0;">ChallengeAnswerMessage</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span>
<span style="color: #d0d0d0;">HandlerName</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">manual</span>
<span style="color: #d0d0d0;">HandlerHandleDate</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">3/13/2017</span> <span style="color: #d0d0d0;">9</span><span style="background-color: #e3d2d2; color: #a61717;">:</span><span style="color: #d0d0d0;">51</span><span style="background-color: #e3d2d2; color: #a61717;">:</span><span style="color: #d0d0d0;">49</span> <span style="color: #d0d0d0;">PM</span>
<span style="color: #d0d0d0;">HandlerCleanUpDate</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span>
<span style="color: #d0d0d0;">SubmitDate</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">3/13/2017</span> <span style="color: #d0d0d0;">9</span><span style="background-color: #e3d2d2; color: #a61717;">:</span><span style="color: #d0d0d0;">56</span><span style="background-color: #e3d2d2; color: #a61717;">:</span><span style="color: #d0d0d0;">02</span> <span style="color: #d0d0d0;">PM</span>
<span style="color: #d0d0d0;">SubmitResponse</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">{StatusCode,</span> <span style="color: #d0d0d0;">Headers,</span> <span style="color: #d0d0d0;">Links,</span> <span style="color: #d0d0d0;">RawContent...}</span>
<span style="color: #d0d0d0;">PS</span> <span style="color: #d0d0d0;">C</span><span style="background-color: #e3d2d2; color: #a61717;">:</span><span style="color: #d0d0d0;">\Vault></span> <span style="color: #24909d;">Update-ACMEIdentifier</span> <span style="color: #d0d0d0;">mserver3</span>
<span style="color: #d0d0d0;">IdentifierPart</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">ACMESharp.Messages.IdentifierPart</span>
<span style="color: #d0d0d0;">IdentifierType</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">dns</span>
<span style="color: #d0d0d0;">Identifier</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">mserver1.dataascii.com</span>
<span style="color: #d0d0d0;">Uri</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">https</span><span style="background-color: #e3d2d2; color: #a61717;">:</span><span style="color: #d0d0d0;">//acme-v01.api.letsencrypt.org/acme/authz/KoKUWptBR61KJWILZObPmZLxsK_nDhhQvKQRYrqMg5w</span>
<span style="color: #d0d0d0;">Status</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">valid</span>
<span style="color: #d0d0d0;">Expires</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">5/12/2017</span> <span style="color: #d0d0d0;">2</span><span style="background-color: #e3d2d2; color: #a61717;">:</span><span style="color: #d0d0d0;">56</span><span style="background-color: #e3d2d2; color: #a61717;">:</span><span style="color: #d0d0d0;">05</span> <span style="color: #d0d0d0;">PM</span>
<span style="color: #d0d0d0;">Challenges</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">{,</span> <span style="color: #d0d0d0;">,</span> <span style="color: #d0d0d0;">}</span>
<span style="color: #d0d0d0;">Combinations</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">{2,</span> <span style="color: #d0d0d0;">0,</span> <span style="color: #d0d0d0;">1}</span>
<span style="color: #d0d0d0;">PS</span> <span style="color: #d0d0d0;">C</span><span style="background-color: #e3d2d2; color: #a61717;">:</span><span style="color: #d0d0d0;">\Vault></span> <span style="color: #24909d;">New-ACMECertificate</span> <span style="color: #d0d0d0;">mserver3</span> <span style="color: #d0d0d0;">-Generate</span> <span style="color: #d0d0d0;">-Alias</span> <span style="color: #d0d0d0;">cert3</span>
<span style="color: #d0d0d0;">Id</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">89f6cf64-62b0-4295-ac77-b0213e11e598</span>
<span style="color: #6ab825; font-weight: bold;">Alias</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">cert3</span>
<span style="color: #d0d0d0;">Label</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span>
<span style="color: #d0d0d0;">Memo</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span>
<span style="color: #d0d0d0;">IdentifierRef</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">2180716e-74fc-4086-9849-0dfd43654b46</span>
<span style="color: #d0d0d0;">IdentifierDns</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">mserver1.dataascii.com</span>
<span style="color: #d0d0d0;">AlternativeIdentifierDns</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span>
<span style="color: #d0d0d0;">KeyPemFile</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span>
<span style="color: #d0d0d0;">CsrPemFile</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span>
<span style="color: #d0d0d0;">GenerateDetailsFile</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">89f6cf64-62b0-4295-ac77-b0213e11e598-gen.json</span>
<span style="color: #d0d0d0;">CertificateRequest</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span>
<span style="color: #d0d0d0;">CrtPemFile</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span>
<span style="color: #d0d0d0;">CrtDerFile</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span>
<span style="color: #d0d0d0;">IssuerSerialNumber</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span>
<span style="color: #d0d0d0;">SerialNumber</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span>
<span style="color: #d0d0d0;">Thumbprint</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span>
<span style="color: #d0d0d0;">Signature</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span>
<span style="color: #d0d0d0;">SignatureAlgorithm</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span>
<span style="color: #d0d0d0;">PS</span> <span style="color: #d0d0d0;">C</span><span style="background-color: #e3d2d2; color: #a61717;">:</span><span style="color: #d0d0d0;">\Vault></span> <span style="color: #d0d0d0;">Submit-ACMECertificate</span> <span style="color: #d0d0d0;">cert3</span>
<span style="color: #d0d0d0;">Id</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">89f6cf64-62b0-4295-ac77-b0213e11e598</span>
<span style="color: #6ab825; font-weight: bold;">Alias</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">cert3</span>
<span style="color: #d0d0d0;">Label</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span>
<span style="color: #d0d0d0;">Memo</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span>
<span style="color: #d0d0d0;">IdentifierRef</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">2180716e-74fc-4086-9849-0dfd43654b46</span>
<span style="color: #d0d0d0;">IdentifierDns</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">mserver1.dataascii.com</span>
<span style="color: #d0d0d0;">AlternativeIdentifierDns</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span>
<span style="color: #d0d0d0;">KeyPemFile</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">89f6cf64-62b0-4295-ac77-b0213e11e598-key.pem</span>
<span style="color: #d0d0d0;">CsrPemFile</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">89f6cf64-62b0-4295-ac77-b0213e11e598-csr.pem</span>
<span style="color: #d0d0d0;">GenerateDetailsFile</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">89f6cf64-62b0-4295-ac77-b0213e11e598-gen.json</span>
<span style="color: #d0d0d0;">CertificateRequest</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">ACMESharp.CertificateRequest</span>
<span style="color: #d0d0d0;">CrtPemFile</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">89f6cf64-62b0-4295-ac77-b0213e11e598-crt.pem</span>
<span style="color: #d0d0d0;">CrtDerFile</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">89f6cf64-62b0-4295-ac77-b0213e11e598-crt.der</span>
<span style="color: #d0d0d0;">IssuerSerialNumber</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span>
<span style="color: #d0d0d0;">SerialNumber</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">03822465656D5211595DE8205941A599B322</span>
<span style="color: #d0d0d0;">Thumbprint</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">5A1F028B8EFB83EB79B2246885753B0A0ED34670</span>
<span style="color: #d0d0d0;">Signature</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">5A1F028B8EFB83EB79B2246885753B0A0ED34670</span>
<span style="color: #d0d0d0;">SignatureAlgorithm</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">sha256RSA</span>
<span style="color: #d0d0d0;">PS</span> <span style="color: #d0d0d0;">C</span><span style="background-color: #e3d2d2; color: #a61717;">:</span><span style="color: #d0d0d0;">\Vault></span> <span style="color: #24909d;">Get-ACMECertificate</span> <span style="color: #d0d0d0;">cert3</span> <span style="color: #d0d0d0;">-ExportKeyPEM</span> <span style="color: #ed9d13;">"c:\Vault\cert3.key.pem"</span>
<span style="color: #d0d0d0;">Id</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">89f6cf64-62b0-4295-ac77-b0213e11e598</span>
<span style="color: #6ab825; font-weight: bold;">Alias</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">cert3</span>
<span style="color: #d0d0d0;">Label</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span>
<span style="color: #d0d0d0;">Memo</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span>
<span style="color: #d0d0d0;">IdentifierRef</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">2180716e-74fc-4086-9849-0dfd43654b46</span>
<span style="color: #d0d0d0;">IdentifierDns</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">mserver1.dataascii.com</span>
<span style="color: #d0d0d0;">AlternativeIdentifierDns</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span>
<span style="color: #d0d0d0;">KeyPemFile</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">89f6cf64-62b0-4295-ac77-b0213e11e598-key.pem</span>
<span style="color: #d0d0d0;">CsrPemFile</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">89f6cf64-62b0-4295-ac77-b0213e11e598-csr.pem</span>
<span style="color: #d0d0d0;">GenerateDetailsFile</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">89f6cf64-62b0-4295-ac77-b0213e11e598-gen.json</span>
<span style="color: #d0d0d0;">CertificateRequest</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">ACMESharp.CertificateRequest</span>
<span style="color: #d0d0d0;">CrtPemFile</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">89f6cf64-62b0-4295-ac77-b0213e11e598-crt.pem</span>
<span style="color: #d0d0d0;">CrtDerFile</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">89f6cf64-62b0-4295-ac77-b0213e11e598-crt.der</span>
<span style="color: #d0d0d0;">IssuerSerialNumber</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span>
<span style="color: #d0d0d0;">SerialNumber</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">03822465656D5211595DE8205941A599B322</span>
<span style="color: #d0d0d0;">Thumbprint</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">5A1F028B8EFB83EB79B2246885753B0A0ED34670</span>
<span style="color: #d0d0d0;">Signature</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">5A1F028B8EFB83EB79B2246885753B0A0ED34670</span>
<span style="color: #d0d0d0;">SignatureAlgorithm</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">sha256RSA</span>
<span style="color: #d0d0d0;">PS</span> <span style="color: #d0d0d0;">C</span><span style="background-color: #e3d2d2; color: #a61717;">:</span><span style="color: #d0d0d0;">\Vault></span> <span style="color: #24909d;">Get-ACMECertificate</span> <span style="color: #d0d0d0;">cert3</span> <span style="color: #d0d0d0;">-ExportCsrPEM</span> <span style="color: #ed9d13;">"c:\Vault\cert3.csr.pem"</span>
<span style="color: #d0d0d0;">Id</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">89f6cf64-62b0-4295-ac77-b0213e11e598</span>
<span style="color: #6ab825; font-weight: bold;">Alias</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">cert3</span>
<span style="color: #d0d0d0;">Label</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span>
<span style="color: #d0d0d0;">Memo</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span>
<span style="color: #d0d0d0;">IdentifierRef</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">2180716e-74fc-4086-9849-0dfd43654b46</span>
<span style="color: #d0d0d0;">IdentifierDns</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">mserver1.dataascii.com</span>
<span style="color: #d0d0d0;">AlternativeIdentifierDns</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span>
<span style="color: #d0d0d0;">KeyPemFile</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">89f6cf64-62b0-4295-ac77-b0213e11e598-key.pem</span>
<span style="color: #d0d0d0;">CsrPemFile</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">89f6cf64-62b0-4295-ac77-b0213e11e598-csr.pem</span>
<span style="color: #d0d0d0;">GenerateDetailsFile</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">89f6cf64-62b0-4295-ac77-b0213e11e598-gen.json</span>
<span style="color: #d0d0d0;">CertificateRequest</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">ACMESharp.CertificateRequest</span>
<span style="color: #d0d0d0;">CrtPemFile</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">89f6cf64-62b0-4295-ac77-b0213e11e598-crt.pem</span>
<span style="color: #d0d0d0;">CrtDerFile</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">89f6cf64-62b0-4295-ac77-b0213e11e598-crt.der</span>
<span style="color: #d0d0d0;">IssuerSerialNumber</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span>
<span style="color: #d0d0d0;">SerialNumber</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">03822465656D5211595DE8205941A599B322</span>
<span style="color: #d0d0d0;">Thumbprint</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">5A1F028B8EFB83EB79B2246885753B0A0ED34670</span>
<span style="color: #d0d0d0;">Signature</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">5A1F028B8EFB83EB79B2246885753B0A0ED34670</span>
<span style="color: #d0d0d0;">SignatureAlgorithm</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">sha256RSA</span>
<span style="color: #d0d0d0;">PS</span> <span style="color: #d0d0d0;">C</span><span style="background-color: #e3d2d2; color: #a61717;">:</span><span style="color: #d0d0d0;">\Vault></span> <span style="color: #24909d;">Get-ACMECertificate</span> <span style="color: #d0d0d0;">cert3</span> <span style="color: #d0d0d0;">-ExportCertificatePEM</span> <span style="color: #ed9d13;">"c:\Vault\cert3.crt.pem"</span> <span style="color: #d0d0d0;">-ExportCertificateDER</span> <span style="color: #ed9d13;">"c:\Vault\ce</span>
<span style="color: #ed9d13;">rt3.crt"</span>
<span style="color: #d0d0d0;">Id</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">89f6cf64-62b0-4295-ac77-b0213e11e598</span>
<span style="color: #6ab825; font-weight: bold;">Alias</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">cert3</span>
<span style="color: #d0d0d0;">Label</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span>
<span style="color: #d0d0d0;">Memo</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span>
<span style="color: #d0d0d0;">IdentifierRef</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">2180716e-74fc-4086-9849-0dfd43654b46</span>
<span style="color: #d0d0d0;">IdentifierDns</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">mserver1.dataascii.com</span>
<span style="color: #d0d0d0;">AlternativeIdentifierDns</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span>
<span style="color: #d0d0d0;">KeyPemFile</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">89f6cf64-62b0-4295-ac77-b0213e11e598-key.pem</span>
<span style="color: #d0d0d0;">CsrPemFile</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">89f6cf64-62b0-4295-ac77-b0213e11e598-csr.pem</span>
<span style="color: #d0d0d0;">GenerateDetailsFile</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">89f6cf64-62b0-4295-ac77-b0213e11e598-gen.json</span>
<span style="color: #d0d0d0;">CertificateRequest</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">ACMESharp.CertificateRequest</span>
<span style="color: #d0d0d0;">CrtPemFile</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">89f6cf64-62b0-4295-ac77-b0213e11e598-crt.pem</span>
<span style="color: #d0d0d0;">CrtDerFile</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">89f6cf64-62b0-4295-ac77-b0213e11e598-crt.der</span>
<span style="color: #d0d0d0;">IssuerSerialNumber</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span>
<span style="color: #d0d0d0;">SerialNumber</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">03822465656D5211595DE8205941A599B322</span>
<span style="color: #d0d0d0;">Thumbprint</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">5A1F028B8EFB83EB79B2246885753B0A0ED34670</span>
<span style="color: #d0d0d0;">Signature</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">5A1F028B8EFB83EB79B2246885753B0A0ED34670</span>
<span style="color: #d0d0d0;">SignatureAlgorithm</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">sha256RSA</span>
<span style="color: #d0d0d0;">PS</span> <span style="color: #d0d0d0;">C</span><span style="background-color: #e3d2d2; color: #a61717;">:</span><span style="color: #d0d0d0;">\Vault></span> <span style="color: #24909d;">Get-ACMECertificate</span> <span style="color: #d0d0d0;">cert3</span> <span style="color: #d0d0d0;">-ExportIssuerPEM</span> <span style="color: #ed9d13;">"c:\Vault\cert3-issuer.crt.pem"</span> <span style="color: #d0d0d0;">-ExportIssuerDER</span> <span style="color: #ed9d13;">"c:\Vault\cert3</span>
<span style="color: #ed9d13;">-issuer.crt"</span>
<span style="color: #24909d;">Get-ACMECertificate</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">Issuer</span> <span style="color: #d0d0d0;">certificate</span> <span style="color: #d0d0d0;">hasn</span><span style="color: #ed9d13;">'t been resolved</span>
<span style="color: #ed9d13;">At line:1 char:1</span>
<span style="color: #ed9d13;">+ Get-ACMECertificate cert3 -ExportIssuerPEM "c:\Vault\cert3-issuer.crt.pem" -Expo ...</span>
<span style="color: #ed9d13;">+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~</span>
<span style="color: #ed9d13;"> + CategoryInfo : NotSpecified: (:) [Get-ACMECertificate], InvalidOperationException</span>
<span style="color: #ed9d13;"> + FullyQualifiedErrorId : System.InvalidOperationException,ACMESharp.POSH.GetCertificate</span>
<span style="color: #ed9d13;">PS C:\Vault> Update-ACMECertificate cert3</span>
<span style="color: #ed9d13;">Id : 89f6cf64-62b0-4295-ac77-b0213e11e598</span>
<span style="color: #ed9d13;">Alias : cert3</span>
<span style="color: #ed9d13;">Label :</span>
<span style="color: #ed9d13;">Memo :</span>
<span style="color: #ed9d13;">IdentifierRef : 2180716e-74fc-4086-9849-0dfd43654b46</span>
<span style="color: #ed9d13;">IdentifierDns : mserver1.dataascii.com</span>
<span style="color: #ed9d13;">AlternativeIdentifierDns :</span>
<span style="color: #ed9d13;">KeyPemFile : 89f6cf64-62b0-4295-ac77-b0213e11e598-key.pem</span>
<span style="color: #ed9d13;">CsrPemFile : 89f6cf64-62b0-4295-ac77-b0213e11e598-csr.pem</span>
<span style="color: #ed9d13;">GenerateDetailsFile : 89f6cf64-62b0-4295-ac77-b0213e11e598-gen.json</span>
<span style="color: #ed9d13;">CertificateRequest : ACMESharp.CertificateRequest</span>
<span style="color: #ed9d13;">CrtPemFile : 89f6cf64-62b0-4295-ac77-b0213e11e598-crt.pem</span>
<span style="color: #ed9d13;">CrtDerFile : 89f6cf64-62b0-4295-ac77-b0213e11e598-crt.der</span>
<span style="color: #ed9d13;">IssuerSerialNumber : 0A0141420000015385736A0B85ECA708</span>
<span style="color: #ed9d13;">SerialNumber : 03822465656D5211595DE8205941A599B322</span>
<span style="color: #ed9d13;">Thumbprint : 5A1F028B8EFB83EB79B2246885753B0A0ED34670</span>
<span style="color: #ed9d13;">Signature : 5A1F028B8EFB83EB79B2246885753B0A0ED34670</span>
<span style="color: #ed9d13;">SignatureAlgorithm : sha256RSA</span>
<span style="color: #ed9d13;">PS C:\Vault> Get-ACMECertificate cert3 -ExportIssuerPEM "c:\Vault\cert3-issuer.crt.pem" -ExportIssuerDER "c:\Vault\cert3</span>
<span style="color: #ed9d13;">-issuer.crt"</span>
<span style="color: #ed9d13;">Id : 89f6cf64-62b0-4295-ac77-b0213e11e598</span>
<span style="color: #ed9d13;">Alias : cert3</span>
<span style="color: #ed9d13;">Label :</span>
<span style="color: #ed9d13;">Memo :</span>
<span style="color: #ed9d13;">IdentifierRef : 2180716e-74fc-4086-9849-0dfd43654b46</span>
<span style="color: #ed9d13;">IdentifierDns : mserver1.dataascii.com</span>
<span style="color: #ed9d13;">AlternativeIdentifierDns :</span>
<span style="color: #ed9d13;">KeyPemFile : 89f6cf64-62b0-4295-ac77-b0213e11e598-key.pem</span>
<span style="color: #ed9d13;">CsrPemFile : 89f6cf64-62b0-4295-ac77-b0213e11e598-csr.pem</span>
<span style="color: #ed9d13;">GenerateDetailsFile : 89f6cf64-62b0-4295-ac77-b0213e11e598-gen.json</span>
<span style="color: #ed9d13;">CertificateRequest : ACMESharp.CertificateRequest</span>
<span style="color: #ed9d13;">CrtPemFile : 89f6cf64-62b0-4295-ac77-b0213e11e598-crt.pem</span>
<span style="color: #ed9d13;">CrtDerFile : 89f6cf64-62b0-4295-ac77-b0213e11e598-crt.der</span>
<span style="color: #ed9d13;">IssuerSerialNumber : 0A0141420000015385736A0B85ECA708</span>
<span style="color: #ed9d13;">SerialNumber : 03822465656D5211595DE8205941A599B322</span>
<span style="color: #ed9d13;">Thumbprint : 5A1F028B8EFB83EB79B2246885753B0A0ED34670</span>
<span style="color: #ed9d13;">Signature : 5A1F028B8EFB83EB79B2246885753B0A0ED34670</span>
<span style="color: #ed9d13;">SignatureAlgorithm : sha256RSA</span>
<span style="color: #ed9d13;">PS C:\Vault> Get-ACMECertificate cert3 -ExportPkcs12 "c:\Vault\cert3.pfx"</span>
<span style="color: #ed9d13;">Id : 89f6cf64-62b0-4295-ac77-b0213e11e598</span>
<span style="color: #ed9d13;">Alias : cert3</span>
<span style="color: #ed9d13;">Label :</span>
<span style="color: #ed9d13;">Memo :</span>
<span style="color: #ed9d13;">IdentifierRef : 2180716e-74fc-4086-9849-0dfd43654b46</span>
<span style="color: #ed9d13;">IdentifierDns : mserver1.dataascii.com</span>
<span style="color: #ed9d13;">AlternativeIdentifierDns :</span>
<span style="color: #ed9d13;">KeyPemFile : 89f6cf64-62b0-4295-ac77-b0213e11e598-key.pem</span>
<span style="color: #ed9d13;">CsrPemFile : 89f6cf64-62b0-4295-ac77-b0213e11e598-csr.pem</span>
<span style="color: #ed9d13;">GenerateDetailsFile : 89f6cf64-62b0-4295-ac77-b0213e11e598-gen.json</span>
<span style="color: #ed9d13;">CertificateRequest : ACMESharp.CertificateRequest</span>
<span style="color: #ed9d13;">CrtPemFile : 89f6cf64-62b0-4295-ac77-b0213e11e598-crt.pem</span>
<span style="color: #ed9d13;">CrtDerFile : 89f6cf64-62b0-4295-ac77-b0213e11e598-crt.der</span>
<span style="color: #ed9d13;">IssuerSerialNumber : 0A0141420000015385736A0B85ECA708</span>
<span style="color: #ed9d13;">SerialNumber : 03822465656D5211595DE8205941A599B322</span>
<span style="color: #ed9d13;">Thumbprint : 5A1F028B8EFB83EB79B2246885753B0A0ED34670</span>
<span style="color: #ed9d13;">Signature : 5A1F028B8EFB83EB79B2246885753B0A0ED34670</span>
<span style="color: #ed9d13;">SignatureAlgorithm : sha256RSA</span>
<span style="color: #ed9d13;">PS C:\Vault> Get-ACMECertificate cert3 -ExportPkcs12 "c:\Vault\cert3-hard.pfx" -CertificatePassword '</span><span style="color: #d0d0d0;">password</span><span style="background-color: #e3d2d2; color: #a61717;">'</span>
<span style="color: #d0d0d0;">Id</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">89f6cf64-62b0-4295-ac77-b0213e11e598</span>
<span style="color: #6ab825; font-weight: bold;">Alias</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">cert3</span>
<span style="color: #d0d0d0;">Label</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span>
<span style="color: #d0d0d0;">Memo</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span>
<span style="color: #d0d0d0;">IdentifierRef</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">2180716e-74fc-4086-9849-0dfd43654b46</span>
<span style="color: #d0d0d0;">IdentifierDns</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">mserver1.dataascii.com</span>
<span style="color: #d0d0d0;">AlternativeIdentifierDns</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span>
<span style="color: #d0d0d0;">KeyPemFile</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">89f6cf64-62b0-4295-ac77-b0213e11e598-key.pem</span>
<span style="color: #d0d0d0;">CsrPemFile</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">89f6cf64-62b0-4295-ac77-b0213e11e598-csr.pem</span>
<span style="color: #d0d0d0;">GenerateDetailsFile</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">89f6cf64-62b0-4295-ac77-b0213e11e598-gen.json</span>
<span style="color: #d0d0d0;">CertificateRequest</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">ACMESharp.CertificateRequest</span>
<span style="color: #d0d0d0;">CrtPemFile</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">89f6cf64-62b0-4295-ac77-b0213e11e598-crt.pem</span>
<span style="color: #d0d0d0;">CrtDerFile</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">89f6cf64-62b0-4295-ac77-b0213e11e598-crt.der</span>
<span style="color: #d0d0d0;">IssuerSerialNumber</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">0A0141420000015385736A0B85ECA708</span>
<span style="color: #d0d0d0;">SerialNumber</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">03822465656D5211595DE8205941A599B322</span>
<span style="color: #d0d0d0;">Thumbprint</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">5A1F028B8EFB83EB79B2246885753B0A0ED34670</span>
<span style="color: #d0d0d0;">Signature</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">5A1F028B8EFB83EB79B2246885753B0A0ED34670</span>
<span style="color: #d0d0d0;">SignatureAlgorithm</span> <span style="background-color: #e3d2d2; color: #a61717;">:</span> <span style="color: #d0d0d0;">sha256RSA</span>
<span style="color: #d0d0d0;">PS</span> <span style="color: #d0d0d0;">C</span><span style="background-color: #e3d2d2; color: #a61717;">:</span><span style="color: #d0d0d0;">\Vault></span>
</pre>
</div>
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />พี่นัทhttp://www.blogger.com/profile/14291406590594822628noreply@blogger.com0tag:blogger.com,1999:blog-2488490746136830746.post-85546991518287458752017-01-24T17:33:00.001+07:002017-01-24T17:38:58.782+07:00Execute SSH on PowerShell<br />
<br />
<!-- HTML generated using hilite.me --><br />
<div style="background: #111111; border-width: 0.1em 0.1em 0.1em 0.8em; border: solid gray; overflow: auto; padding: 0.2em 0.6em; width: auto;">
<pre style="line-height: 125%; margin: 0;"><span style="color: white;">PS</span> <span style="color: white;">C:\Users\Administrator\Documents></span> <span style="color: white;">Find-Module</span> <span style="color: white;">Posh-SSH</span>
<span style="color: white;">Version</span> <span style="color: white;">Name</span> <span style="color: white;">Type</span> <span style="color: white;">Repository</span> <span style="color: white;">Description</span>
<span style="color: white;">-------</span> <span style="color: white;">----</span> <span style="color: white;">----</span> <span style="color: white;">----------</span> <span style="color: white;">-----------</span>
<span style="color: white;">1.7.7</span> <span style="color: white;">Posh-SSH</span> <span style="color: white;">Module</span> <span style="color: white;">PSGallery</span> <span style="color: white;">Provide</span> <span style="color: white;">SSH</span> <span style="color: white;">functionality</span> <span style="color: #fb660a; font-weight: bold;">for</span> <span style="color: white;">executi...</span>
<span style="color: white;">PS</span> <span style="color: white;">C:\Users\Administrator\Documents></span> <span style="color: white;">Install-Module</span> <span style="color: white;">Posh-SSH</span>
<span style="color: white;">Untrusted</span> <span style="color: white;">repository</span>
<span style="color: white;">You</span> <span style="color: white;">are</span> <span style="color: white;">installing</span> <span style="color: white;">the</span> <span style="color: white;">modules</span> <span style="color: white;">from</span> <span style="color: white;">an</span> <span style="color: white;">untrusted</span> <span style="color: white;">repository.</span> <span style="color: #fb660a; font-weight: bold;">If</span> <span style="color: white;">you</span> <span style="color: white;">trust</span> <span style="color: white;">this</span> <span style="color: white;">repository,</span> <span style="color: white;">change</span> <span style="color: white;">its</span> <span style="color: white;">InstallationPolic</span>
<span style="color: white;">y</span> <span style="color: white;">value</span> <span style="color: white;">by</span> <span style="color: white;">running</span> <span style="color: white;">the</span> <span style="color: white;">Set-PSRepository</span> <span style="color: white;">cmdlet.</span> <span style="color: white;">Are</span> <span style="color: white;">you</span> <span style="color: white;">sure</span> <span style="color: white;">you</span> <span style="color: white;">want</span> <span style="color: white;">to</span> <span style="color: white;">install</span> <span style="color: white;">the</span> <span style="color: white;">modules</span> <span style="color: white;">from</span> <span style="color: #0086d2;">'PSGallery'</span><span style="color: white;">?</span>
<span style="color: #0086d2;">[Y] Yes [A] Yes to All [N] No [L]</span> <span style="color: white;">No</span> <span style="color: white;">to</span> <span style="color: white;">All</span> <span style="color: white;">[?]</span> <span style="color: white;">Help</span> <span style="color: white;">(</span><span style="color: #fb660a; font-weight: bold;">default</span> <span style="color: white;">is</span> <span style="color: #0086d2;">'Y'</span><span style="color: white;">):</span>
<span style="color: white;">PS</span> <span style="color: white;">C:\Users\Administrator\Documents></span> <span style="color: white;">New-SSHSession</span> <span style="color: white;">-ComputerName</span> <span style="color: #0086d2;">"172.18.3.20"</span> <span style="color: white;">-Credential</span> <span style="color: white;">(Get-Credential)</span>
<span style="color: white;">Windows</span> <span style="color: white;">PowerShell</span> <span style="color: white;">Credential</span> <span style="color: white;">Request</span> <span style="color: white;">:</span> <span style="color: white;">cmdlet</span> <span style="color: white;">Get-Credential</span> <span style="color: white;">at</span> <span style="color: white;">command</span> <span style="color: white;">pipeline</span> <span style="color: #fb660a; font-weight: bold;">position</span> <span style="color: white;">1</span>
<span style="color: white;">Warning:</span> <span style="color: white;">This</span> <span style="color: white;">credential</span> <span style="color: white;">is</span> <span style="color: white;">being</span> <span style="color: white;">requested</span> <span style="color: white;">by</span> <span style="color: white;">a</span> <span style="color: white;">script</span> <span style="color: white;">or</span> <span style="color: white;">application</span> <span style="color: white;">on</span> <span style="color: white;">the</span> <span style="color: white;">STATBSPAPP</span> <span style="color: white;">remote</span> <span style="color: white;">computer.</span> <span style="color: white;">Enter</span> <span style="color: white;">your</span> <span style="color: white;">cr</span>
<span style="color: white;">edentials</span> <span style="color: white;">only</span> <span style="color: #fb660a; font-weight: bold;">if</span> <span style="color: white;">you</span> <span style="color: white;">trust</span> <span style="color: white;">the</span> <span style="color: white;">remote</span> <span style="color: white;">computer</span> <span style="color: white;">and</span> <span style="color: white;">the</span> <span style="color: white;">application</span> <span style="color: white;">or</span> <span style="color: white;">script</span> <span style="color: white;">requesting</span> <span style="color: white;">it.</span>
<span style="color: white;">Supply</span> <span style="color: white;">values</span> <span style="color: #fb660a; font-weight: bold;">for</span> <span style="color: white;">the</span> <span style="color: white;">following</span> <span style="color: white;">parameters:</span>
<span style="color: white;">Credential:</span>
<span style="color: white;">Enter</span> <span style="color: white;">your</span> <span style="color: white;">user</span> <span style="color: white;">name:</span> <span style="color: white;">root</span>
<span style="color: white;">Enter</span> <span style="color: white;">your</span> <span style="color: white;">password:</span> <span style="color: white;">*****************</span>
<span style="color: white;">Server</span> <span style="color: white;">SSH</span> <span style="color: white;">Fingerprint</span>
<span style="color: #fb660a; font-weight: bold;">Do</span> <span style="color: white;">you</span> <span style="color: white;">want</span> <span style="color: white;">to</span> <span style="color: white;">trust</span> <span style="color: white;">the</span> <span style="color: white;">fingerprint</span> <span style="color: white;">d4:d0:f9:cd:ad:22:ee:da:25:b2:2b:50:45:81:de:5a</span>
<span style="color: white;">[]</span> <span style="color: white;">Y</span> <span style="color: white;">[]</span> <span style="color: white;">N</span> <span style="color: white;">[?]</span> <span style="color: white;">Help</span> <span style="color: white;">(</span><span style="color: #fb660a; font-weight: bold;">default</span> <span style="color: white;">is</span> <span style="color: #0086d2;">'N'</span><span style="color: white;">):</span> <span style="color: white;">Y</span>
<span style="color: white;">SessionId</span> <span style="color: white;">Host</span> <span style="color: white;">Connected</span>
<span style="color: white;">---------</span> <span style="color: white;">----</span> <span style="color: white;">---------</span>
<span style="color: white;">0</span> <span style="color: white;">172.18.3.20</span> <span style="color: white;">True</span>
<span style="color: white;">PS</span> <span style="color: white;">C:\Users\Administrator\Documents></span> <span style="color: white;">Invoke-SSHCommand</span> <span style="color: white;">-Index</span> <span style="color: white;">0</span> <span style="color: white;">-Command</span> <span style="color: #0086d2;">"uname"</span>
<span style="color: white;">Host</span> <span style="color: white;">:</span> <span style="color: white;">172.18.3.20</span>
<span style="color: white;">Output</span> <span style="color: white;">:</span> <span style="color: white;">{SunOS}</span>
<span style="color: white;">ExitStatus</span> <span style="color: white;">:</span> <span style="color: white;">0</span>
<span style="color: white;">PS</span> <span style="color: white;">C:\Users\Administrator\Documents></span> <span style="color: white;">Invoke-SSHCommand</span> <span style="color: white;">-Index</span> <span style="color: white;">0</span> <span style="color: white;">-Command</span> <span style="color: #0086d2;">"su - oracle -c id -u -n"</span>
<span style="color: white;">Host</span> <span style="color: white;">:</span> <span style="color: white;">172.18.3.20</span>
<span style="color: white;">Output</span> <span style="color: white;">:</span> <span style="color: white;">{uid=901(oracle)</span> <span style="color: white;">gid=900(dba)}</span>
<span style="color: white;">ExitStatus</span> <span style="color: white;">:</span> <span style="color: white;">0</span>
</pre>
</div>
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />พี่นัทhttp://www.blogger.com/profile/14291406590594822628noreply@blogger.com0tag:blogger.com,1999:blog-2488490746136830746.post-68667773364703550082017-01-18T22:15:00.003+07:002017-01-20T13:25:09.400+07:00Windows Remote Management and PowerShell Web Access<!-- HTML generated using hilite.me --><br />
<br />
<br />
<br />
<div style="background: rgb(248, 248, 248); border: solid gray; overflow: auto; padding: 0.2em 0.6em; width: auto;">
<pre style="line-height: 125%; margin: 0px;">PS C<span style="border: 1px solid rgb(255, 0, 0);">:</span>\Users\Administrator> dir WSMan<span style="border: 1px solid rgb(255, 0, 0);">:</span>\localhost\Listener\*\Port
WSManConfig<span style="border: 1px solid rgb(255, 0, 0);">:</span> Microsoft.WSMan.Management\WSMan<span style="border: 1px solid rgb(255, 0, 0);">::</span>localhost\Listener\Listener_1084132640
Type Name SourceOfValue Value
---- ---- ------------- -----
System.String Port 5985
PS C<span style="border: 1px solid rgb(255, 0, 0);">:</span>\Users\Administrator> <span style="color: green;">Set-Item</span> WSMan<span style="border: 1px solid rgb(255, 0, 0);">:</span>\localhost\Listener\*\Port 8888
Set the value of the item
This command set the value of the Item.
<span style="color: green; font-weight: bold;">Do</span> you want to <span style="color: green; font-weight: bold;">continue</span>?
<span style="color: #880000;">[Y] Yes [N] No [S]</span> Suspend [?] Help (<span style="color: green; font-weight: bold;">default</span> is <span style="color: #ba2121;">"Y"</span>)<span style="border: 1px solid rgb(255, 0, 0);">:</span>
PS C<span style="border: 1px solid rgb(255, 0, 0);">:</span>\Users\Administrator> dir WSMan<span style="border: 1px solid rgb(255, 0, 0);">:</span>\localhost\Listener\*\Port
WSManConfig<span style="border: 1px solid rgb(255, 0, 0);">:</span> Microsoft.WSMan.Management\WSMan<span style="border: 1px solid rgb(255, 0, 0);">::</span>localhost\Listener\Listener_1892757853
Type Name SourceOfValue Value
---- ---- ------------- -----
System.String Port 8888
PS C<span style="border: 1px solid rgb(255, 0, 0);">:</span>\Users\Administrator>
PS C<span style="border: 1px solid rgb(255, 0, 0);">:</span>\Users\Administrator> <span style="color: green;">Enable-PSRemoting</span>
WinRM Quick Configuration
Running command <span style="color: #ba2121;">"Set-WSManQuickConfig"</span> to enable remote management of this computer by using the Windows Remote
Management (WinRM) service.
This includes<span style="border: 1px solid rgb(255, 0, 0);">:</span>
1. Starting or restarting (<span style="color: green; font-weight: bold;">if</span> already started) the WinRM service
2. Setting the WinRM service startup type to Automatic
3. Creating a listener to accept requests on any IP address
4. Enabling Windows Firewall inbound rule exceptions <span style="color: green; font-weight: bold;">for</span> WS-Management traffic (<span style="color: green; font-weight: bold;">for</span> http only).
<span style="color: green; font-weight: bold;">Do</span> you want to <span style="color: green; font-weight: bold;">continue</span>?
<span style="color: #880000;">[Y] Yes [A] Yes to All [N] No [L] No to All [S]</span> Suspend [?] Help (<span style="color: green; font-weight: bold;">default</span> is <span style="color: #ba2121;">"Y"</span>)<span style="border: 1px solid rgb(255, 0, 0);">:</span>
WinRM is already set up to receive requests on this computer.
WinRM has been updated <span style="color: green; font-weight: bold;">for</span> remote management.
Configured LocalAccountTokenFilterPolicy to grant administrative rights remotely to local users.
Confirm
Are you sure you want to perform this action?
Performing the operation <span style="color: #ba2121;">"Set-PSSessionConfiguration"</span> on target <span style="color: #ba2121;">"Name: microsoft.powershell SDDL:</span>
<span style="color: #ba2121;">O:NSG:BAD:P(A;;GA;;;BA)(A;;GA;;;RM)S:P(AU;FA;GA;;;WD)(AU;SA;GXGW;;;WD). This lets selected users remotely run Windows</span>
<span style="color: #ba2121;">PowerShell commands on this computer."</span>.
<span style="color: #880000;">[Y] Yes [A] Yes to All [N] No [L] No to All [S]</span> Suspend [?] Help (<span style="color: green; font-weight: bold;">default</span> is <span style="color: #ba2121;">"Y"</span>)<span style="border: 1px solid rgb(255, 0, 0);">:</span>
Confirm
Are you sure you want to perform this action?
Performing the operation <span style="color: #ba2121;">"Set-PSSessionConfiguration"</span> on target <span style="color: #ba2121;">"Name: microsoft.powershell.workflow SDDL:</span>
<span style="color: #ba2121;">O:NSG:BAD:P(A;;GA;;;BA)(A;;GA;;;RM)S:P(AU;FA;GA;;;WD)(AU;SA;GXGW;;;WD). This lets selected users remotely run Windows</span>
<span style="color: #ba2121;">PowerShell commands on this computer."</span>.
<span style="color: #880000;">[Y] Yes [A] Yes to All [N] No [L] No to All [S]</span> Suspend [?] Help (<span style="color: green; font-weight: bold;">default</span> is <span style="color: #ba2121;">"Y"</span>)<span style="border: 1px solid rgb(255, 0, 0);">:</span>
Confirm
Are you sure you want to perform this action?
Performing the operation <span style="color: #ba2121;">"Set-PSSessionConfiguration"</span> on target <span style="color: #ba2121;">"Name: microsoft.powershell32 SDDL:</span>
<span style="color: #ba2121;">O:NSG:BAD:P(A;;GA;;;BA)(A;;GA;;;RM)S:P(AU;FA;GA;;;WD)(AU;SA;GXGW;;;WD). This lets selected users remotely run Windows</span>
<span style="color: #ba2121;">PowerShell commands on this computer."</span>.
<span style="color: #880000;">[Y] Yes [A] Yes to All [N] No [L] No to All [S]</span> Suspend [?] Help (<span style="color: green; font-weight: bold;">default</span> is <span style="color: #ba2121;">"Y"</span>)<span style="border: 1px solid rgb(255, 0, 0);">:</span>
Confirm
Are you sure you want to perform this action?
Performing the operation <span style="color: #ba2121;">"Set-PSSessionConfiguration"</span> on target <span style="color: #ba2121;">"Name: microsoft.windows.servermanagerworkflows SDDL:</span>
<span style="color: #ba2121;">O:NSG:BAD:P(A;;GA;;;BA)(A;;GA;;;IU)S:P(AU;FA;GA;;;WD)(AU;SA;GXGW;;;WD). This lets selected users remotely run Windows</span>
<span style="color: #ba2121;">PowerShell commands on this computer."</span>.
<span style="color: #880000;">[Y] Yes [A] Yes to All [N] No [L] No to All [S]</span> Suspend [?] Help (<span style="color: green; font-weight: bold;">default</span> is <span style="color: #ba2121;">"Y"</span>)<span style="border: 1px solid rgb(255, 0, 0);">:</span>
PS C<span style="border: 1px solid rgb(255, 0, 0);">:</span>\Users\Administrator> dir WSMan<span style="border: 1px solid rgb(255, 0, 0);">:</span>\localhost\Listener\*\Port
WSManConfig<span style="border: 1px solid rgb(255, 0, 0);">:</span> Microsoft.WSMan.Management\WSMan<span style="border: 1px solid rgb(255, 0, 0);">::</span>localhost\Listener\Listener_1892757853
Type Name SourceOfValue Value
---- ---- ------------- -----
System.String Port 8888
</pre>
</div>
<br />
Check WinRM running with new ports<br />
<!-- HTML generated using hilite.me --><br />
<div style="background: #f8f8f8; border-width: 0.1em 0.1em 0.1em 0.8em; border: solid gray; overflow: auto; padding: 0.2em 0.6em; width: auto;">
<pre style="line-height: 125%; margin: 0;">PS C<span style="border: 1px solid #FF0000;">:</span>\Users\Administrator> netstat -aon | findstr <span style="color: #ba2121;">"8888"</span>
TCP 0.0.0.0<span style="border: 1px solid #FF0000;">:</span>8888 0.0.0.0<span style="border: 1px solid #FF0000;">:</span>0 LISTENING 4
TCP [<span style="border: 1px solid #FF0000;">::</span>]<span style="border: 1px solid #FF0000;">:</span>8888 [<span style="border: 1px solid #FF0000;">::</span>]<span style="border: 1px solid #FF0000;">:</span>0 LISTENING 4
PS C<span style="border: 1px solid #FF0000;">:</span>\Users\Administrator>
</pre>
</div>
<br />
Install Web Access<br />
<br />
<div style="background: #f8f8f8; border-width: 0.1em 0.1em 0.1em 0.8em; border: solid gray; overflow: auto; padding: 0.2em 0.6em; width: auto;">
<pre style="line-height: 125%; margin: 0;">PS C<span style="border: 1px solid #FF0000;">:</span>\Users\Administrator> Install-WindowsFeature <span style="border: 1px solid #FF0000;">–</span>Name WindowsPowerShellWebAccess -ComputerName MSERVER1 -IncludeManage
mentTools -Restart
Success Restart Needed Exit Code Feature Result
------- -------------- --------- --------------
True No Success {ASP.NET 4.5, Application Development, ASP...
WARNING<span style="border: 1px solid #FF0000;">:</span> <span style="color: green; font-weight: bold;">For</span> more information about how to finish installing and configuring Windows PowerShell Web Access, see
http<span style="border: 1px solid #FF0000;">:</span>//go.microsoft.com/fwlink/<span style="color: green; font-weight: bold;">?</span>LinkID=221050.
PS C<span style="border: 1px solid #FF0000;">:</span>\Users\Administrator>
</pre>
</div>
<br />
Config gateway<br />
<br />
<!-- HTML generated using hilite.me --><br />
<div style="background: #f8f8f8; border-width: 0.1em 0.1em 0.1em 0.8em; border: solid gray; overflow: auto; padding: 0.2em 0.6em; width: auto;">
<pre style="line-height: 125%; margin: 0;">PS C<span style="border: 1px solid #FF0000;">:</span>\Users\Administrator> Install-PswaWebApplication -UseTestCertificate
WARNING<span style="border: 1px solid #FF0000;">:</span> Using a test certificate <span style="color: green; font-weight: bold;">in</span> a production environment is not recommended <span style="color: green; font-weight: bold;">for</span> security reasons. This certificate
should be used only <span style="color: green; font-weight: bold;">for</span> internal testing of Windows PowerShell Web Access. The test certificate expires <span style="color: green; font-weight: bold;">in</span> 90 days.
Creating application pool pswa_pool...
Name State Applications
---- ----- ------------
pswa_pool Started
Creating web application pswa...
Path <span style="border: 1px solid #FF0000;">:</span> /pswa
ApplicationPool <span style="border: 1px solid #FF0000;">:</span> pswa_pool
EnabledProtocols <span style="border: 1px solid #FF0000;">:</span> http
PhysicalPath <span style="border: 1px solid #FF0000;">:</span> C<span style="border: 1px solid #FF0000;">:</span>\Windows\Web\PowerShellWebAccess\wwwroot
Creating self-signed certificate...
Creating HTTPS binding...
PS C<span style="border: 1px solid #FF0000;">:</span>\Users\Administrator>
</pre>
</div>
<br />
Add authorization rule<br />
<br />
<!-- HTML generated using hilite.me --><br />
<div style="background: #f8f8f8; border-width: 0.1em 0.1em 0.1em 0.8em; border: solid gray; overflow: auto; padding: 0.2em 0.6em; width: auto;">
<pre style="line-height: 125%; margin: 0;">PS C<span style="border: 1px solid #FF0000;">:</span>\Users\Administrator> hostname
mserver1
PS C<span style="border: 1px solid #FF0000;">:</span>\Users\Administrator> <span style="color: green;">Add-PswaAuthorizationRule</span> <span style="border: 1px solid #FF0000;">–</span>UserName MSERVER1\Administrator -ComputerName MSERVER1 -Configurat
ionName microsoft.powershell
Id RuleName User Destination ConfigurationName
-- -------- ---- ----------- -----------------
0 Rule 0 MSERVER1\Administrator MSERVER1 microsoft.powershell
PS C<span style="border: 1px solid #FF0000;">:</span>\Users\Administrator> <span style="color: green;">Add-PswaAuthorizationRule</span> <span style="border: 1px solid #FF0000;">–</span>UserName MSERVER1\CloudAdmin -ComputerName MSERVER1 -Configuration
Name microsoft.powershell
Id RuleName User Destination ConfigurationName
-- -------- ---- ----------- -----------------
1 Rule 1 MSERVER1\CloudAdmin MSERVER1 microsoft.powershell
PS C<span style="border: 1px solid #FF0000;">:</span>\Users\Administrator> <span style="color: green;">Get-PswaAuthorizationRule</span>
Id RuleName User Destination ConfigurationName
-- -------- ---- ----------- -----------------
0 Rule 0 mserver1\administrator MSERVER1 microsoft.powershell
1 Rule 1 mserver1\cloudadmin MSERVER1 microsoft.powershell
PS C<span style="border: 1px solid #FF0000;">:</span>\Users\Administrator>
</pre>
</div>
<br />
OR allow all user and configuration with<br />
<br />
<!-- HTML generated using hilite.me --><br />
<div style="background: #f8f8f8; border-width: 0.1em 0.1em 0.1em 0.8em; border: solid gray; overflow: auto; padding: 0.2em 0.6em; width: auto;">
<pre style="line-height: 125%; margin: 0;">PS C<span style="border: 1px solid #FF0000;">:</span>\Users\Administrator> <span style="color: green;">Add-PswaAuthorizationRule</span> <span style="border: 1px solid #FF0000;">–</span>UserName * -ComputerName * -ConfigurationName *
-- OR --
PS C<span style="border: 1px solid #FF0000;">:</span>\Users\Administrator> <span style="color: green;">Add-PswaAuthorizationRule</span> * * *
</pre>
</div>
<br />
<br />
<br />
<br />
Bingo !!!.<br />
<br />
Open url => https://xx.xx.xx.xx/pswa<br />
<br />
<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgvHCTkKDVKM5wCmnNn4kQaEcDaiE7eqzBjRserxW89B9q6JrSKwyhndDqWI6NosTguHiEzUr2E6s8ExNMEEDhp6AumQbgan19Df1wZe0FOOFF4clHVPGgglSnNTG6WBpm3eReg7K6_XUIC/s1600/Selection_008.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgvHCTkKDVKM5wCmnNn4kQaEcDaiE7eqzBjRserxW89B9q6JrSKwyhndDqWI6NosTguHiEzUr2E6s8ExNMEEDhp6AumQbgan19Df1wZe0FOOFF4clHVPGgglSnNTG6WBpm3eReg7K6_XUIC/s1600/Selection_008.jpg" /></a></div>
<br />
<br />
Welcome to PS session like below<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj1yxGPZKDFDpU-6Dbe6bkR7viBbOXq-b4lEtmrELxFJ2XcpVHMJbkDNnx6KcF5o-SHMJwJHqdDLIQZDxakRkskfRJsToc2sm-KfguuM0DndPDHvxH-5noBWDX0Pi8iJDX3gDAcUS0zlfBG/s1600/Selection_009.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj1yxGPZKDFDpU-6Dbe6bkR7viBbOXq-b4lEtmrELxFJ2XcpVHMJbkDNnx6KcF5o-SHMJwJHqdDLIQZDxakRkskfRJsToc2sm-KfguuM0DndPDHvxH-5noBWDX0Pi8iJDX3gDAcUS0zlfBG/s1600/Selection_009.jpg" /></a></div>
<br />
<br />
<br />
Additional step for remote to machine in difference domain or non-domain manner.<br />
<br />
Add trusted host<br />
<br />
<!-- HTML generated using hilite.me --><br />
<div style="background: #f8f8f8; border-width: 0.1em 0.1em 0.1em 0.8em; border: solid gray; overflow: auto; padding: 0.2em 0.6em; width: auto;">
<pre style="line-height: 125%; margin: 0;">PS C<span style="border: 1px solid #FF0000;">:</span>\Users\Administrator> <span style="color: green;">Get-Item</span> WSMan<span style="border: 1px solid #FF0000;">:</span>\localhost\Client\TrustedHosts
WSManConfig<span style="border: 1px solid #FF0000;">:</span> Microsoft.WSMan.Management\WSMan<span style="border: 1px solid #FF0000;">::</span>localhost\Client
Type Name SourceOfValue Value
---- ---- ------------- -----
System.String TrustedHosts STATBSPDB,172.19.3.13
PS C<span style="border: 1px solid #FF0000;">:</span>\Users\Administrator> <span style="color: green;">Set-Item</span> WSMan<span style="border: 1px solid #FF0000;">:</span>\localhost\Client\TrustedHosts -Value STATBBI -Force -Concatenate
PS C<span style="border: 1px solid #FF0000;">:</span>\Users\Administrator> <span style="color: green;">Set-Item</span> WSMan<span style="border: 1px solid #FF0000;">:</span>\localhost\Client\TrustedHosts -Value 172.19.2.31 -Force -Concatenate
PS C<span style="border: 1px solid #FF0000;">:</span>\Users\Administrator> <span style="color: green;">Get-Item</span> WSMan<span style="border: 1px solid #FF0000;">:</span>\localhost\Client\TrustedHosts
WSManConfig<span style="border: 1px solid #FF0000;">:</span> Microsoft.WSMan.Management\WSMan<span style="border: 1px solid #FF0000;">::</span>localhost\Client
Type Name SourceOfValue Value
---- ---- ------------- -----
System.String TrustedHosts STATBSPDB,172.19.3.13,STATBBI,172.19.2.31
</pre>
</div>
<br />
Test from local machine (gateway) when run Get-Credential, a windows is pop-up then enter a credential like "STATBBI\Administrator" and password of remote server.<br />
<br />
<!-- HTML generated using hilite.me --><br />
<div style="background: #f8f8f8; border-width: 0.1em 0.1em 0.1em 0.8em; border: solid gray; overflow: auto; padding: 0.2em 0.6em; width: auto;">
<pre style="line-height: 125%; margin: 0;">PS C<span style="border: 1px solid #FF0000;">:</span>\Users\Administrator> <span style="color: #19177c;">$C</span> = <span style="color: green;">Get-Credential</span>
cmdlet <span style="color: green;">Get-Credential</span> at command pipeline <span style="color: green; font-weight: bold;">position</span> 1
Supply values <span style="color: green; font-weight: bold;">for</span> the following parameters<span style="border: 1px solid #FF0000;">:</span>
Credential
PS C<span style="border: 1px solid #FF0000;">:</span>\Users\Administrator> <span style="color: green;">Enter-PSSession</span> -ComputerName STATBBI -Credential <span style="color: #19177c;">$C</span>
<span style="color: #880000;">[STATBBI]</span><span style="border: 1px solid #FF0000;">:</span> PS C<span style="border: 1px solid #FF0000;">:</span>\Users\Administrator\Documents>
<span style="color: #880000;">[STATBBI]</span><span style="border: 1px solid #FF0000;">:</span> PS C<span style="border: 1px solid #FF0000;">:</span>\Users\Administrator\Documents>
<span style="color: #880000;">[STATBBI]</span><span style="border: 1px solid #FF0000;">:</span> PS C<span style="border: 1px solid #FF0000;">:</span>\Users\Administrator\Documents> hostname
STATBBI
</pre>
</div>
<br />
After that try to access from web by enter an info like below.<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi-dL4PZirb1UaCBT3c1kVPoxNudPDhsYtaOxfPzrkelr0XW5ug0nGbtKhWyjbgfCk3GA-dp4TUBjjyjE7QLmi72qkreX1CV65iQktPnilBpSxagmBcqFGKa5cU2zjWdrQnzcdUUXNkNHIn/s1600/Selection_012.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEi-dL4PZirb1UaCBT3c1kVPoxNudPDhsYtaOxfPzrkelr0XW5ug0nGbtKhWyjbgfCk3GA-dp4TUBjjyjE7QLmi72qkreX1CV65iQktPnilBpSxagmBcqFGKa5cU2zjWdrQnzcdUUXNkNHIn/s1600/Selection_012.jpg" /></a></div>
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />พี่นัทhttp://www.blogger.com/profile/14291406590594822628noreply@blogger.com0tag:blogger.com,1999:blog-2488490746136830746.post-29889992249911998562017-01-15T13:00:00.002+07:002017-01-29T00:35:18.627+07:00Windows PowerShell Cmdlets<br />
<!-- HTML generated using hilite.me --><br />
<div style="background: #f8f8f8; border-width: 0.1em 0.1em 0.1em 0.8em; border: solid gray; overflow: auto; padding: 0.2em 0.6em; width: auto;">
<pre style="line-height: 125%; margin: 0;"><span style="color: green;">Get-Service -RequiredServices xxx</span>
<span style="color: green;">Get-Service</span> | <span style="color: green;">Where-Object</span> {<span style="color: #19177c;">$_</span>.displayname <span style="color: #666666;">-like</span> <span style="color: #ba2121;">"*Web*"</span>}
<span style="color: green;">Get-Service</span> | <span style="color: green;">Sort-Object</span> status,displayname
<span style="color: green;">Get-Service</span> | <span style="color: green;">Where-Object</span> {<span style="color: #19177c;">$_</span>.status <span style="color: #666666;">-eq</span> <span style="color: #ba2121;">"running"</span>}
<span style="color: green;">Get-Service</span> | <span style="color: green;">Where-Object</span> {<span style="color: #19177c;">$_</span>.status <span style="color: #666666;">-eq</span> <span style="color: #ba2121;">"running"</span> <span style="color: #666666;">-and</span> <span style="color: #19177c;">$_</span>.displayname <span style="color: #666666;">-like</span> <span style="color: #ba2121;">"*Application*"</span>}
<pre style="color: #333333; line-height: 16.25px;"><span style="color: green;">Get-WmiObject</span> win32_service | <span style="color: green;">Where-Object</span> {<span style="color: #19177c;">$_</span>.displayname <span style="color: #666666;">-like</span> <span style="color: #ba2121;">"SQL Server (MSSQLSERVER)"</span>} | <span style="color: green;">format-list</span> *</pre>
<span style="color: green;">Get-Process</span>
<span style="color: green;">Stop-Process</span> -processname notepad
<span style="color: green;">Start-Service</span> btwdins
<span style="color: green;">Stop-Service</span> btwdins
<span style="color: green;">Restart-Service</span> btwdins,alerter
<span style="color: green;">Stop-Computer</span> -Force
<span style="color: green;">Stop-Computer</span> -ComputerName <span style="color: #ba2121;">"Server01"</span>, <span style="color: #ba2121;">"Server02"</span>, <span style="color: #ba2121;">"localhost"</span>
<span style="color: green;">Stop-Computer</span> <span style="border: 1px solid #FF0000;">–</span>computer DC1 <span style="border: 1px solid #FF0000;">–</span>Credential nwtraders\administrator
<span style="color: green;">Restart-Computer</span> -WhatIf
<span style="color: green;">Restart-Computer</span> <span style="color: #ba2121;">"server01"</span>,<span style="color: #ba2121;">"server02"</span>,<span style="color: #ba2121;">"server03"</span>
<span style="color: green;">Get-EventLog</span> system -newest 5 | <span style="color: green;">Sort-Object</span> eventid -descending
<pre style="color: #333333; line-height: 16.25px;"><pre style="line-height: 16.25px;"><span style="color: green;">Get-EventLog</span> -LogName system -entrytype Error, Warning -After (<span style="color: green;">Get-Date</span>).AddDays(-3)</pre>
</pre>
<pre style="line-height: 125%; margin: 0;"><pre style="color: #333333; line-height: 16.25px;"><pre style="line-height: 16.25px;"><span style="color: green;">Get-EventLog</span> -LogName system -entrytype warning -After (<span style="color: green;">Get-Date</span>).AddDays(-3) | <span style="color: green;">Where-Object</span> {<span style="color: #19177c;">$_</span>.Source <span style="color: #666666;">-ne</span> <span style="color: #ba2121;">"disk"</span>}</pre>
</pre>
<pre style="color: #333333; line-height: 16.25px;"><span style="color: green;">Get-EventLog</span> -Index 18308517 -Logname System | <span style="color: green;">Format-List</span></pre>
</pre>
PS C<span style="border: 1px solid #FF0000;">:</span>\temp\PowerShell2> echo > test.ps1
cmdlet <span style="color: green;">Write-Output</span> at command pipeline <span style="color: green; font-weight: bold;">position</span> 1
Supply values <span style="color: green; font-weight: bold;">for</span> the following parameters<span style="border: 1px solid #FF0000;">:</span>
InputObject[0]<span style="border: 1px solid #FF0000;">:</span> <span style="color: green;">Get-EventLog</span> system -newest 5 | <span style="color: green;">Sort-Object</span> eventid -descending
InputObject[1]<span style="border: 1px solid #FF0000;">:</span> <span style="color: green;">Get-Process</span>
InputObject[2]<span style="border: 1px solid #FF0000;">:</span> <span style="color: green;">Get-Service</span>
InputObject[3]<span style="border: 1px solid #FF0000;">:</span> </pre>
<pre style="line-height: 125%; margin: 0;">PS C<span style="border: 1px solid #FF0000;">:</span>\temp\PowerShell2> type test.ps1</pre>
<pre style="line-height: 125%; margin: 0;"><span style="color: green;">Get-EventLog</span> system -newest 5 | <span style="color: green;">Sort-Object</span> eventid -descending
<span style="color: green;">Get-Process</span>
<span style="color: green;">Get-Service</span>
PS C<span style="border: 1px solid #FF0000;">:</span>\temp\PowerShell2> powershell -ExecutionPolicy ByPass <span style="color: #666666;">-File</span> test.ps1
</pre>
</div>
<br />
<u>File Operation PowerShell</u><br />
<!-- HTML generated using hilite.me --><br />
<div style="background: #f8f8f8; border-width: 0.1em 0.1em 0.1em 0.8em; border: solid gray; overflow: auto; padding: 0.2em 0.6em; width: auto;">
<pre style="line-height: 125%; margin: 0;">PS C<span style="border: 1px solid #FF0000;">:</span>\SysInternals\sendmail>
net use \\STATBSPDB\C$ password /USER<span style="border: 1px solid #FF0000;">:</span>STATBSPDB\administrator
The command completed successfully.
PS C<span style="border: 1px solid #FF0000;">:</span>\SysInternals\sendmail>
<span style="color: green;">Copy-Item</span> \\STATBSPDB\c$\SysInternals\sendmail\EventDetail.txt .
PS C<span style="border: 1px solid #FF0000;">:</span>\SysInternals\sendmail>
dir
Directory<span style="border: 1px solid #FF0000;">:</span> C<span style="border: 1px solid #FF0000;">:</span>\SysInternals\sendmail
Mode LastWriteTime Length Name
---- ------------- ------ ----
-a--- 9/29/2009 11<span style="border: 1px solid #FF0000;">:</span>02 AM 13589 CHANGELOG.txt
-a--- 1/28/2017 11<span style="border: 1px solid #FF0000;">:</span>00 PM 5 close-wait.txt
-a--- 1/12/2017 1<span style="border: 1px solid #FF0000;">:</span>16 PM 782 dbmem.sql
-a--- 1/28/2017 11<span style="border: 1px solid #FF0000;">:</span>01 PM 798 dbmem.txt
-a--- 1/28/2017 11<span style="border: 1px solid #FF0000;">:</span>20 PM 121856 EventDetail.txt
-a--- 12/7/2016 3<span style="border: 1px solid #FF0000;">:</span>15 PM 45501 eventlast3hrs.txt
-a--- 1/28/2017 11<span style="border: 1px solid #FF0000;">:</span>00 PM 255618 eventlast3hrs.xml
-a--- 9/29/2009 11<span style="border: 1px solid #FF0000;">:</span>03 AM 4701 README-BR.txt
-a--- 9/29/2009 11<span style="border: 1px solid #FF0000;">:</span>03 AM 8193 README.txt
-a--- 9/29/2009 10<span style="border: 1px solid #FF0000;">:</span>49 AM 1531966 sendEmail.exe
-a--- 9/29/2009 1<span style="border: 1px solid #FF0000;">:</span>46 PM 82448 sendEmail.pl
-a--- 1/12/2017 3<span style="border: 1px solid #FF0000;">:</span>02 PM 895 sendmail - Copy.cmd
-a--- 1/20/2017 4<span style="border: 1px solid #FF0000;">:</span>29 PM 907 sendmail.cmd
-a--- 1/12/2017 2<span style="border: 1px solid #FF0000;">:</span>57 PM 0 sqlcmd.log
-a--- 9/29/2009 11<span style="border: 1px solid #FF0000;">:</span>03 AM 1256 TODO.txt
-a--- 5/13/2016 11<span style="border: 1px solid #FF0000;">:</span>57 AM 66888 typeperf.htm
-a--- 1/28/2017 11<span style="border: 1px solid #FF0000;">:</span>00 PM 3594 typeperf.out
-a--- 5/13/2016 12<span style="border: 1px solid #FF0000;">:</span>22 PM 1170 typeperf.txt
PS C<span style="border: 1px solid #FF0000;">:</span>\SysInternals\sendmail>
<span style="color: green;">Copy-Item</span> \\STATBSPDB\c$\SysInternals\sendmail\EventDetail_App.txt .
PS C<span style="border: 1px solid #FF0000;">:</span>\SysInternals\sendmail>
dir
PS C<span style="border: 1px solid #FF0000;">:</span>\SysInternals\sendmail>
<span style="color: green;">Invoke-WebRequest</span> -Uri <span style="color: #ba2121;">"http://www.7-zip.org/a/7z1604-x64.msi"</span> -OutFile <span style="color: #ba2121;">"C:\SysInternals\7z1604-x64.msi"</span>
PS C<span style="border: 1px solid #FF0000;">:</span>\SysInternals\sendmail>
<span style="color: green;">Start-Process</span> <span style="color: #ba2121;">"C:\SysInternals\7z1604-x64.msi"</span> /qn -Wait
PS C<span style="border: 1px solid #FF0000;">:</span>\SysInternals\sendmail>
cd <span style="color: #ba2121;">'C:\Program Files\7-Zip'</span>
PS C<span style="border: 1px solid #FF0000;">:</span>\Program Files\7-Zip>
.\7z.exe a C<span style="border: 1px solid #FF0000;">:</span>\SysInternals\sendmail\EventDetail.zip C<span style="border: 1px solid #FF0000;">:</span>\SysInternals\sendmail\EventDetail*
7-Zip [64] 16.04 <span style="border: 1px solid #FF0000;">:</span> Copyright (c) 1999-2016 Igor Pavlov <span style="border: 1px solid #FF0000;">:</span> 2016-10-04
Scanning the drive<span style="border: 1px solid #FF0000;">:</span>
2 files, 29325476 bytes (28 MiB)
Creating archive<span style="border: 1px solid #FF0000;">:</span> C<span style="border: 1px solid #FF0000;">:</span>\SysInternals\sendmail\EventDetail.zip
Items to compress<span style="border: 1px solid #FF0000;">:</span> 2
Files read from disk<span style="border: 1px solid #FF0000;">:</span> 2
Archive size<span style="border: 1px solid #FF0000;">:</span> 482093 bytes (471 KiB)
Everything is Ok
PS C<span style="border: 1px solid #FF0000;">:</span>\Program Files\7-Zip>
cd C<span style="border: 1px solid #FF0000;">:</span>\SysInternals\sendmail\
PS C<span style="border: 1px solid #FF0000;">:</span>\SysInternals\sendmail>
dir
Directory<span style="border: 1px solid #FF0000;">:</span> C<span style="border: 1px solid #FF0000;">:</span>\SysInternals\sendmail
Mode LastWriteTime Length Name
---- ------------- ------ ----
-a--- 9/29/2009 11<span style="border: 1px solid #FF0000;">:</span>02 AM 13589 CHANGELOG.txt
-a--- 1/29/2017 12<span style="border: 1px solid #FF0000;">:</span>00 AM 5 close-wait.txt
-a--- 1/12/2017 1<span style="border: 1px solid #FF0000;">:</span>16 PM 782 dbmem.sql
-a--- 1/29/2017 12<span style="border: 1px solid #FF0000;">:</span>01 AM 798 dbmem.txt
-a--- 1/28/2017 11<span style="border: 1px solid #FF0000;">:</span>20 PM 121856 EventDetail.txt
-a--- 1/29/2017 12<span style="border: 1px solid #FF0000;">:</span>29 AM 482093 EventDetail.zip
-a--- 1/28/2017 11<span style="border: 1px solid #FF0000;">:</span>33 PM 29203620 EventDetail_App.txt
-a--- 12/7/2016 3<span style="border: 1px solid #FF0000;">:</span>15 PM 45501 eventlast3hrs.txt
-a--- 1/29/2017 12<span style="border: 1px solid #FF0000;">:</span>00 AM 260753 eventlast3hrs.xml
-a--- 9/29/2009 11<span style="border: 1px solid #FF0000;">:</span>03 AM 4701 README-BR.txt
-a--- 9/29/2009 11<span style="border: 1px solid #FF0000;">:</span>03 AM 8193 README.txt
-a--- 9/29/2009 10<span style="border: 1px solid #FF0000;">:</span>49 AM 1531966 sendEmail.exe
-a--- 9/29/2009 1<span style="border: 1px solid #FF0000;">:</span>46 PM 82448 sendEmail.pl
-a--- 1/12/2017 3<span style="border: 1px solid #FF0000;">:</span>02 PM 895 sendmail - Copy.cmd
-a--- 1/20/2017 4<span style="border: 1px solid #FF0000;">:</span>29 PM 907 sendmail.cmd
-a--- 1/12/2017 2<span style="border: 1px solid #FF0000;">:</span>57 PM 0 sqlcmd.log
-a--- 9/29/2009 11<span style="border: 1px solid #FF0000;">:</span>03 AM 1256 TODO.txt
-a--- 5/13/2016 11<span style="border: 1px solid #FF0000;">:</span>57 AM 66888 typeperf.htm
-a--- 1/29/2017 12<span style="border: 1px solid #FF0000;">:</span>00 AM 3595 typeperf.out
-a--- 5/13/2016 12<span style="border: 1px solid #FF0000;">:</span>22 PM 1170 typeperf.txt
PS C<span style="border: 1px solid #FF0000;">:</span>\SysInternals\sendmail>
.\sendEmail -o -a EventDetail.zip <span style="color: #666666;">-f</span> spadmin<span style="color: #19177c;">@nso</span>.go.th -t nutthaphon<span style="color: #19177c;">@gmail</span>.com -s mailgw.nso.go.th<span style="border: 1px solid #FF0000;">:</span>25 -u <span style="color: #ba2121;">"Event Viewer"</span>
-m <span style="color: #ba2121;">"FYI"</span>
Jan 29 00<span style="border: 1px solid #FF0000;">:</span>30<span style="border: 1px solid #FF0000;">:</span>30 statbspapp sendEmail.exe[213848]<span style="border: 1px solid #FF0000;">:</span> Email was sent successfully!
PS C<span style="border: 1px solid #FF0000;">:</span>\SysInternals\sendmail>
</pre>
</div>
<br />
<a href="https://www.simple-talk.com/sql/database-administration/practical-powershell-for-sql-server-developers-and-dbas-part-1/" target="_blank">SQL Server PowerShell</a><br />
<br />
First of all, enable scripting restriction then load sqlps module.<br />
<br />
<!-- HTML generated using hilite.me --><br />
<div style="background: #f8f8f8; border-width: 0.1em 0.1em 0.1em 0.8em; border: solid gray; overflow: auto; padding: 0.2em 0.6em; width: auto;">
<pre style="line-height: 125%; margin: 0;">PS C<span style="border: 1px solid #FF0000;">:</span>\Users\Administrator\Documents> <span style="color: green;">Set-ExecutionPolicy</span> RemoteSigned
PS C<span style="border: 1px solid #FF0000;">:</span>\Users\Administrator\Documents> <span style="color: green;">Import-Module</span> sqlps -DisableNameChecking
PS SQLSERVER<span style="border: 1px solid #FF0000;">:</span>\>
</pre>
</div>
<br />
Try it<br />
<!-- HTML generated using hilite.me --><br />
<div style="background: #f8f8f8; border-width: 0.1em 0.1em 0.1em 0.8em; border: solid gray; overflow: auto; padding: 0.2em 0.6em; width: auto;">
<pre style="line-height: 125%; margin: 0;"><pre style="color: #333333; line-height: 16.25px;">PS SQLSERVER<span style="border: 1px solid rgb(255, 0, 0);">:</span>\> <span style="color: green;">Invoke-Sqlcmd</span> -Query <span style="color: #ba2121;">"SELECT GETDATE() AS TimeOfQuery;"</span> -ServerInstance <span style="color: #ba2121;">"STATBSPDB"</span> <span style="border: 1px solid rgb(255, 0, 0);">–</span>Username <span style="color: #ba2121;">"sa"</span> <span style="border: 1px solid rgb(255, 0, 0);">–</span>Password <span style="color: #ba2121;">"p@ssw0rd</span>
<span style="color: #ba2121;">"</span></pre>
<pre style="color: #333333; line-height: 16.25px;"><span style="color: #ba2121;">-- or --</span></pre>
<pre style="color: #333333; line-height: 16.25px;"><span style="color: #ba2121;">
</span></pre>
</pre>
<pre style="line-height: 125%; margin: 0;">PS SQLSERVER<span style="border: 1px solid #FF0000;">:</span>\> <span style="color: green;">Invoke-Sqlcmd</span> -ServerInstance . -Database master -Query <span style="color: #ba2121;">"select GetDate()"</span>
Column1
-------
19-Jan-17 2<span style="border: 1px solid #FF0000;">:</span>54<span style="border: 1px solid #FF0000;">:</span>07 PM
</pre>
</div>
<br />
<!-- HTML generated using hilite.me --><br />
<div style="background: #f8f8f8; border-width: 0.1em 0.1em 0.1em 0.8em; border: solid gray; overflow: auto; padding: 0.2em 0.6em; width: auto;">
<pre style="line-height: 125%; margin: 0;">cd SQLSERVER<span style="border: 1px solid #FF0000;">:</span>\SQL\STATBSPDB\<span style="color: green; font-weight: bold;">DEFAULT</span>\Databases
PS SQLSERVER<span style="border: 1px solid #FF0000;">:</span>\SQL\STATBSPDB\<span style="color: green; font-weight: bold;">DEFAULT</span>\Databases>
cd WSS_Content
PS SQLSERVER<span style="border: 1px solid #FF0000;">:</span>\SQL\STATBSPDB\<span style="color: green; font-weight: bold;">DEFAULT</span>\Databases\WSS_Content>
<span style="color: green;">Invoke-Sqlcmd</span> -Query <span style="color: #ba2121;">"SELECT DB_NAME() as [Database]"</span>
WARNING<span style="border: 1px solid #FF0000;">:</span> Using provider context. Server = STATBSPDB, Database = WSS_Content.
Database
--------
WSS_Content
PS SQLSERVER<span style="border: 1px solid #FF0000;">:</span>\SQL\STATBSPDB\<span style="color: green; font-weight: bold;">DEFAULT</span>\Databases\WSS_Content>
</pre>
</div>
<br />
<!-- HTML generated using hilite.me --><br />
<div style="background: #f8f8f8; border-width: 0.1em 0.1em 0.1em 0.8em; border: solid gray; overflow: auto; padding: 0.2em 0.6em; width: auto;">
<pre style="line-height: 125%; margin: 0;">PS SQLSERVER<span style="border: 1px solid #FF0000;">:</span>\SQL\STATBSPDB\<span style="color: green; font-weight: bold;">DEFAULT</span>\Databases\WSS_Content>
ls tables -Force
Schema Name Created
------ ---- -------
dbo AllDocs 02-May-14 3<span style="border: 1px solid #FF0000;">:</span>46 PM
dbo AllDocVersions 02-May-14 3<span style="border: 1px solid #FF0000;">:</span>46 PM
dbo AllFileFragments 02-May-14 3<span style="border: 1px solid #FF0000;">:</span>46 PM
dbo AllLinks 02-May-14 3<span style="border: 1px solid #FF0000;">:</span>46 PM
dbo AllLists 02-May-14 3<span style="border: 1px solid #FF0000;">:</span>46 PM
dbo AllListsAux 02-May-14 3<span style="border: 1px solid #FF0000;">:</span>46 PM
dbo AllListsItemCount 02-May-14 3<span style="border: 1px solid #FF0000;">:</span>46 PM
dbo AllListsPlus 02-May-14 3<span style="border: 1px solid #FF0000;">:</span>46 PM
dbo AllListUniqueFields 02-May-14 3<span style="border: 1px solid #FF0000;">:</span>46 PM
dbo AllLookupRelationships 02-May-14 3<span style="border: 1px solid #FF0000;">:</span>46 PM
dbo AllSites 02-May-14 3<span style="border: 1px solid #FF0000;">:</span>46 PM
dbo AllUserData 02-May-14 3<span style="border: 1px solid #FF0000;">:</span>46 PM
dbo AllUserDataJunctions 02-May-14 3<span style="border: 1px solid #FF0000;">:</span>46 PM
dbo AllWebParts 02-May-14 3<span style="border: 1px solid #FF0000;">:</span>46 PM
dbo AllWebs 02-May-14 3<span style="border: 1px solid #FF0000;">:</span>46 PM
</pre>
</div>
<br />
<!-- HTML generated using hilite.me --><br />
<div style="background: #f8f8f8; border-width: 0.1em 0.1em 0.1em 0.8em; border: solid gray; overflow: auto; padding: 0.2em 0.6em; width: auto;">
<pre style="line-height: 125%; margin: 0;">PS SQLSERVER<span style="border: 1px solid #FF0000;">:</span>\SQL\STATBSPDB\<span style="color: green; font-weight: bold;">DEFAULT</span>\Databases\master>
<span style="color: green;">Set-Location</span> SQLSERVER<span style="border: 1px solid #FF0000;">:</span>\SQL\STATBSPDB\<span style="color: green; font-weight: bold;">DEFAULT</span>\Databases\master
PS SQLSERVER<span style="border: 1px solid #FF0000;">:</span>\SQL\STATBSPDB\<span style="color: green; font-weight: bold;">DEFAULT</span>\Databases\master>
sql <span style="color: #ba2121;">"select * from spt_monitor"</span>
WARNING<span style="border: 1px solid #FF0000;">:</span> Using provider context. Server = STATBSPDB, Database = master.
lastrun <span style="border: 1px solid #FF0000;">:</span> 10-Feb-12 9<span style="border: 1px solid #FF0000;">:</span>02<span style="border: 1px solid #FF0000;">:</span>09 PM
cpu_busy <span style="border: 1px solid #FF0000;">:</span> 20
io_busy <span style="border: 1px solid #FF0000;">:</span> 9
idle <span style="border: 1px solid #FF0000;">:</span> 2707
pack_received <span style="border: 1px solid #FF0000;">:</span> 39
pack_sent <span style="border: 1px solid #FF0000;">:</span> 39
connections <span style="border: 1px solid #FF0000;">:</span> 19
pack_errors <span style="border: 1px solid #FF0000;">:</span> 0
total_read <span style="border: 1px solid #FF0000;">:</span> 0
total_write <span style="border: 1px solid #FF0000;">:</span> 0
total_errors <span style="border: 1px solid #FF0000;">:</span> 0
PS SQLSERVER<span style="border: 1px solid #FF0000;">:</span>\SQL\STATBSPDB\<span style="color: green; font-weight: bold;">DEFAULT</span>\Databases\master>
</pre>
</div>
<br />
<br />
SharePoint PowerShell<br />
Manual registration on Windows PowerShell<br />
<br />
<!-- HTML generated using hilite.me --><br />
<div style="background: #f8f8f8; border-width: 0.1em 0.1em 0.1em 0.8em; border: solid gray; overflow: auto; padding: 0.2em 0.6em; width: auto;">
<pre style="line-height: 125%; margin: 0;">PS C<span style="border: 1px solid #FF0000;">:</span>\Users\Administrator> C<span style="border: 1px solid #FF0000;">:</span>\<span style="color: #ba2121;">"Program Files"</span>\<span style="color: #ba2121;">"Common Files"</span>\<span style="color: #ba2121;">"microsoft shared"</span>\<span style="color: #ba2121;">"Web Server Extensions"</span>\15\CONFIG\POWERSHELL\Registration\SharePoint.ps1
</pre>
</div>
<br />
-- OR --<br />
<br />
Add-PSSnapin Microsoft.SharePoint.Powershell<br />
<br />
<br />
Networking PowerShell<br />
<br />
<!-- HTML generated using hilite.me --><br />
<div style="background: #f8f8f8; border-width: 0.1em 0.1em 0.1em 0.8em; border: solid gray; overflow: auto; padding: 0.2em 0.6em; width: auto;">
<pre style="line-height: 125%; margin: 0;"><span style="color: green;">Get-DnsClientCache</span>
<span style="color: green;">Clear-DnsClientCache</span>
<span style="color: green;">Get-DnsClientGlobalSetting</span>
<span style="color: green;">Get-DnsClientServerAddress</span>
<span style="color: green;">Resolve-DnsName</span> xx.yy.zz -server 172.18.1.11
<span style="color: green;">Get-NetIPAddress</span>
<span style="color: green;">Get-NetIPInterface</span>
<span style="color: green;">Get-NetRoute</span>
<span style="color: green;">Get-NetTCPConnection</span>
<span style="color: green;">Get-NetTCPSetting</span>
<span style="color: green;">Get-NetAdapter</span>
<span style="color: green;">Get-NetAdapterAdvancedProperty</span>
<span style="color: green;">Get-NetAdapterStatistics</span>
</pre>
</div>
<br />
<br />
<br />
<br />
<br />
<br />พี่นัทhttp://www.blogger.com/profile/14291406590594822628noreply@blogger.com0tag:blogger.com,1999:blog-2488490746136830746.post-28136304830424063102017-01-10T22:00:00.000+07:002017-01-11T11:24:11.551+07:00Redhat Ansible Tower Installation<br />
<br />
Check correct date time<br />
<!-- HTML generated using hilite.me --><br />
<div style="background: #f8f8f8; border-width: 0.1em 0.1em 0.1em 0.8em; border: solid gray; overflow: auto; padding: 0.2em 0.6em; width: auto;">
<pre style="line-height: 125%; margin: 0;"><span style="color: navy; font-weight: bold;">[root@rhelnode1 jboss]#</span> date
<span style="color: #888888;">อ. 10 ม.ค. 2560 03:04:44 EST</span>
<span style="color: navy; font-weight: bold;">[root@rhelnode1 jboss]#</span> timedatectl
<span style="color: #888888;"> Local time: อ. 2017-01-10 03:06:49 EST</span>
<span style="color: #888888;"> Universal time: อ. 2017-01-10 08:06:49 UTC</span>
<span style="color: #888888;"> RTC time: อ. 2017-01-10 08:06:48</span>
<span style="color: #888888;"> Time zone: America/New_York (EST, -0500)</span>
<span style="color: #888888;"> NTP enabled: yes</span>
<span style="color: #888888;">NTP synchronized: yes</span>
<span style="color: #888888;"> RTC in local TZ: no</span>
<span style="color: #888888;"> DST active: no</span>
<span style="color: #888888;"> Last DST change: DST ended at</span>
<span style="color: #888888;"> อา. 2016-11-06 01:59:59 EDT</span>
<span style="color: #888888;"> อา. 2016-11-06 01:00:00 EST</span>
<span style="color: #888888;"> Next DST change: DST begins (the clock jumps one hour forward) at</span>
<span style="color: #888888;"> อา. 2017-03-12 01:59:59 EST</span>
<span style="color: #888888;"> อา. 2017-03-12 03:00:00 EDT</span>
<span style="color: navy; font-weight: bold;">[root@rhelnode1 jboss]#</span> timedatectl
<span style="color: navy; font-weight: bold;">[root@rhelnode1 jboss]#</span> timedatectl <span style="color: green;">set</span>-ntp yes
<span style="color: navy; font-weight: bold;">[root@rhelnode1 jboss]#</span> timedatectl list-timezones | grep Bangkok
<span style="color: #888888;">Asia/Bangkok</span>
<span style="color: navy; font-weight: bold;">[root@rhelnode1 jboss]#</span>
<span style="color: navy; font-weight: bold;">[root@rhelnode1 jboss]#</span>
<span style="color: navy; font-weight: bold;">[root@rhelnode1 jboss]#</span>
<span style="color: navy; font-weight: bold;">[root@rhelnode1 jboss]#</span> timedatectl <span style="color: green;">set</span>-timezone Asia/Bangkok
<span style="color: navy; font-weight: bold;">[root@rhelnode1 jboss]#</span> date
<span style="color: #888888;">อ. 10 ม.ค. 2560 15:14:10 ICT</span>
<span style="color: navy; font-weight: bold;">[root@rhelnode1 jboss]#</span> timedatectl
<span style="color: #888888;"> Local time: อ. 2017-01-10 15:14:32 ICT</span>
<span style="color: #888888;"> Universal time: อ. 2017-01-10 08:14:32 UTC</span>
<span style="color: #888888;"> RTC time: อ. 2017-01-10 08:14:17</span>
<span style="color: #888888;"> Time zone: Asia/Bangkok (ICT, +0700)</span>
<span style="color: #888888;"> NTP enabled: yes</span>
<span style="color: #888888;">NTP synchronized: yes</span>
<span style="color: #888888;"> RTC in local TZ: no</span>
<span style="color: #888888;"> DST active: n/a</span>
</pre>
</div>
<br />
<br />
Configure Repository Access (<a href="https://access.redhat.com/solutions/912213?tour=6" target="_blank">link</a>) or download from url<br />
Ref.<br />
<a href="https://access.redhat.com/solutions/3358">https://access.redhat.com/solutions/3358</a><br />
<a href="http://fedoraproject.org/wiki/EPEL#How_can_I_use_these_extra_packages.3F">http://fedoraproject.org/wiki/EPEL#How_can_I_use_these_extra_packages.3F</a><br />
<br />
<!-- HTML generated using hilite.me --><br />
<div style="background: #f8f8f8; border-width: 0.1em 0.1em 0.1em 0.8em; border: solid gray; overflow: auto; padding: 0.2em 0.6em; width: auto;">
<pre style="line-height: 125%; margin: 0;"><span style="color: navy; font-weight: bold;">[jboss@rhelnode1 ~]$</span> ls -lt
<span style="color: #888888;">total 736</span>
<span style="color: #888888;">-rw-r--r--. 1 jboss jboss 737047 ม.ค. 10 02:24 ansible-tower-setup-latest.tar.gz</span>
<span style="color: #888888;">-rw-r--r--. 1 jboss jboss 14612 ม.ค. 10 02:23 epel-release-latest-7.noarch.rpm</span>
<span style="color: navy; font-weight: bold;">[jboss@rhelnode1 ~]$</span> sudo yum install epel-release-latest-7.noarch.rpm
<span style="color: #888888;">[sudo] password for jboss: </span>
<span style="color: #888888;">Loaded plugins: search-disabled-repos</span>
<span style="color: #888888;">Examining epel-release-latest-7.noarch.rpm: epel-release-7-8.noarch</span>
<span style="color: #888888;">Marking epel-release-latest-7.noarch.rpm to be installed</span>
<span style="color: #888888;">Resolving Dependencies</span>
<span style="color: #888888;">--> Running transaction check</span>
<span style="color: #888888;">---> Package epel-release.noarch 0:7-8 will be installed</span>
<span style="color: #888888;">--> Finished Dependency Resolution</span>
<span style="color: #888888;">Dependencies Resolved</span>
<span style="color: #888888;">======================================================================================================================================================</span>
<span style="color: #888888;"> Package Arch Version Repository Size</span>
<span style="color: #888888;">======================================================================================================================================================</span>
<span style="color: #888888;">Installing:</span>
<span style="color: #888888;"> epel-release noarch 7-8 /epel-release-latest-7.noarch 24 k</span>
<span style="color: #888888;">Transaction Summary</span>
<span style="color: #888888;">======================================================================================================================================================</span>
<span style="color: #888888;">Install 1 Package</span>
<span style="color: #888888;">Total size: 24 k</span>
<span style="color: #888888;">Installed size: 24 k</span>
<span style="color: #888888;">Is this ok [y/d/N]: y</span>
<span style="color: #888888;">Downloading packages:</span>
<span style="color: #888888;">Running transaction check</span>
<span style="color: #888888;">Running transaction test</span>
<span style="color: #888888;">Transaction test succeeded</span>
<span style="color: #888888;">Running transaction</span>
<span style="color: #888888;"> Installing : epel-release-7-8.noarch 1/1 </span>
<span style="color: #888888;"> Verifying : epel-release-7-8.noarch 1/1 </span>
<span style="color: #888888;">Installed:</span>
<span style="color: #888888;"> epel-release.noarch 0:7-8 </span>
<span style="color: #888888;">Complete!</span>
</pre>
</div>
<br />
<br />
Subscribe to the new Extras channel/repo (<a href="https://access.redhat.com/solutions/912213?tour=6" target="_blank">link</a>)<br />
<br />
<!-- HTML generated using hilite.me --><br />
<div style="background: #f8f8f8; border-width: 0.1em 0.1em 0.1em 0.8em; border: solid gray; overflow: auto; padding: 0.2em 0.6em; width: auto;">
<pre style="line-height: 125%; margin: 0;"><span style="color: navy; font-weight: bold;">[root@rhelnode1 rhn]#</span> subscription-manager register --auto-attach</pre>
<pre style="line-height: 125%; margin: 0;"><span style="color: #888888;">Registering to: subscription.rhsm.redhat.com:443/subscription</span>
<span style="color: #888888;">Username: nutthaphon@gmail.com</span>
<span style="color: #888888;">Password: </span>
<span style="color: #888888;">The system has been registered with ID: 7a8d7e9a-022e-47c8-a5e2-12102c7c4609 </span>
<span style="color: #888888;">Installed Product Current Status:</span>
<span style="color: #888888;">Product Name: Red Hat Enterprise Linux Server</span>
<span style="color: #888888;">Status: Subscribed</span>
<span style="color: navy; font-weight: bold;">
</span></pre>
<pre style="line-height: 125%; margin: 0;"><span style="color: navy; font-weight: bold;">[root@rhelnode1 rhn]#</span> subscription-manager repos --enable rhel-7-server-extras-rpms
</pre>
<pre style="line-height: 125%; margin: 0;"><span style="color: #888888;">Repository 'rhel-7-server-extras-rpms' is enabled for this system.</span>
</pre>
</div>
<br />
<!-- HTML generated using hilite.me --><br />
<div style="background: #f8f8f8; border-width: 0.1em 0.1em 0.1em 0.8em; border: solid gray; overflow: auto; padding: 0.2em 0.6em; width: auto;">
<pre style="line-height: 125%; margin: 0;"><span style="color: navy; font-weight: bold;">[root@rhelnode1 rhn]#</span> yum install ansible
<span style="color: #888888;">Loaded plugins: search-disabled-repos</span>
<span style="color: #888888;">rhel-7-server-extras-rpms | 3.4 kB 00:00:00 </span>
<span style="color: #888888;">rhel-7-server-rpms | 3.5 kB 00:00:00 </span>
<span style="color: #888888;">(1/6): rhel-7-server-extras-rpms/x86_64/group | 104 B 00:00:01 </span>
<span style="color: #888888;">(2/6): rhel-7-server-extras-rpms/x86_64/updateinfo | 120 kB 00:00:01 </span>
<span style="color: #888888;">(3/6): rhel-7-server-extras-rpms/x86_64/primary_db | 162 kB 00:00:00 </span>
<span style="color: #888888;">(4/6): rhel-7-server-rpms/7Server/x86_64/group | 701 kB 00:00:01 </span>
<span style="color: #888888;">(5/6): rhel-7-server-rpms/7Server/x86_64/updateinfo | 1.8 MB 00:00:03 </span>
<span style="color: #888888;">(6/6): rhel-7-server-rpms/7Server/x86_64/primary_db | 32 MB 00:00:24 </span>
<span style="color: #888888;">Resolving Dependencies</span>
<span style="color: #888888;">--> Running transaction check</span>
<span style="color: #888888;">---> Package ansible.noarch 0:2.2.0.0-4.el7 will be installed</span>
<span style="color: #888888;">--> Processing Dependency: sshpass for package: ansible-2.2.0.0-4.el7.noarch</span>
<span style="color: #888888;">--> Processing Dependency: python-paramiko for package: ansible-2.2.0.0-4.el7.noarch</span>
<span style="color: #888888;">--> Processing Dependency: python-keyczar for package: ansible-2.2.0.0-4.el7.noarch</span>
<span style="color: #888888;">--> Processing Dependency: python-jinja2 for package: ansible-2.2.0.0-4.el7.noarch</span>
<span style="color: #888888;">--> Processing Dependency: python-httplib2 for package: ansible-2.2.0.0-4.el7.noarch</span>
<span style="color: #888888;">--> Running transaction check</span>
<span style="color: #888888;">---> Package python-httplib2.noarch 0:0.7.7-3.el7 will be installed</span>
<span style="color: #888888;">---> Package python-jinja2.noarch 0:2.7.2-2.el7 will be installed</span>
<span style="color: #888888;">--> Processing Dependency: python-babel >= 0.8 for package: python-jinja2-2.7.2-2.el7.noarch</span>
<span style="color: #888888;">--> Processing Dependency: python-markupsafe for package: python-jinja2-2.7.2-2.el7.noarch</span>
<span style="color: #888888;">---> Package python-keyczar.noarch 0:0.71c-2.el7 will be installed</span>
<span style="color: #888888;">--> Processing Dependency: python-pyasn1 for package: python-keyczar-0.71c-2.el7.noarch</span>
<span style="color: #888888;">--> Processing Dependency: python-crypto for package: python-keyczar-0.71c-2.el7.noarch</span>
<span style="color: #888888;">---> Package python2-paramiko.noarch 0:1.16.1-1.el7 will be installed</span>
<span style="color: #888888;">--> Processing Dependency: python2-ecdsa for package: python2-paramiko-1.16.1-1.el7.noarch</span>
<span style="color: #888888;">---> Package sshpass.x86_64 0:1.05-5.el7 will be installed</span>
<span style="color: #888888;">--> Running transaction check</span>
<span style="color: #888888;">---> Package python-babel.noarch 0:0.9.6-8.el7 will be installed</span>
<span style="color: #888888;">---> Package python-markupsafe.x86_64 0:0.11-10.el7 will be installed</span>
<span style="color: #888888;">---> Package python2-crypto.x86_64 0:2.6.1-10.el7 will be installed</span>
<span style="color: #888888;">--> Processing Dependency: libtomcrypt.so.0()(64bit) for package: python2-crypto-2.6.1-10.el7.x86_64</span>
<span style="color: #888888;">---> Package python2-ecdsa.noarch 0:0.13-4.el7 will be installed</span>
<span style="color: #888888;">---> Package python2-pyasn1.noarch 0:0.1.9-7.el7 will be installed</span>
<span style="color: #888888;">--> Running transaction check</span>
<span style="color: #888888;">---> Package libtomcrypt.x86_64 0:1.17-23.el7 will be installed</span>
<span style="color: #888888;">--> Processing Dependency: libtommath >= 0.42.0 for package: libtomcrypt-1.17-23.el7.x86_64</span>
<span style="color: #888888;">--> Processing Dependency: libtommath.so.0()(64bit) for package: libtomcrypt-1.17-23.el7.x86_64</span>
<span style="color: #888888;">--> Running transaction check</span>
<span style="color: #888888;">---> Package libtommath.x86_64 0:0.42.0-4.el7 will be installed</span>
<span style="color: #888888;">--> Finished Dependency Resolution</span>
<span style="color: #888888;">Dependencies Resolved</span>
<span style="color: #888888;">======================================================================================================================================================</span>
<span style="color: #888888;"> Package Arch Version Repository Size</span>
<span style="color: #888888;">======================================================================================================================================================</span>
<span style="color: #888888;">Installing:</span>
<span style="color: #888888;"> ansible noarch 2.2.0.0-4.el7 epel 4.6 M</span>
<span style="color: #888888;">Installing for dependencies:</span>
<span style="color: #888888;"> libtomcrypt x86_64 1.17-23.el7 epel 224 k</span>
<span style="color: #888888;"> libtommath x86_64 0.42.0-4.el7 epel 35 k</span>
<span style="color: #888888;"> python-babel noarch 0.9.6-8.el7 rhel-7-server-rpms 1.4 M</span>
<span style="color: #888888;"> python-httplib2 noarch 0.7.7-3.el7 epel 70 k</span>
<span style="color: #888888;"> python-jinja2 noarch 2.7.2-2.el7 rhel-7-server-extras-rpms 516 k</span>
<span style="color: #888888;"> python-keyczar noarch 0.71c-2.el7 epel 218 k</span>
<span style="color: #888888;"> python-markupsafe x86_64 0.11-10.el7 rhel-7-server-rpms 25 k</span>
<span style="color: #888888;"> python2-crypto x86_64 2.6.1-10.el7 epel 475 k</span>
<span style="color: #888888;"> python2-ecdsa noarch 0.13-4.el7 epel 83 k</span>
<span style="color: #888888;"> python2-paramiko noarch 1.16.1-1.el7 epel 257 k</span>
<span style="color: #888888;"> python2-pyasn1 noarch 0.1.9-7.el7 rhel-7-server-rpms 100 k</span>
<span style="color: #888888;"> sshpass x86_64 1.05-5.el7 epel 21 k</span>
<span style="color: #888888;">Transaction Summary</span>
<span style="color: #888888;">======================================================================================================================================================</span>
<span style="color: #888888;">Install 1 Package (+12 Dependent packages)</span>
<span style="color: #888888;">Total download size: 7.9 M</span>
<span style="color: #888888;">Installed size: 35 M</span>
<span style="color: #888888;">Is this ok [y/d/N]: y</span>
<span style="color: #888888;">Downloading packages:</span>
<span style="color: #888888;">(1/13): libtommath-0.42.0-4.el7.x86_64.rpm | 35 kB 00:00:01 </span>
<span style="color: #888888;">(2/13): libtomcrypt-1.17-23.el7.x86_64.rpm | 224 kB 00:00:01 </span>
<span style="color: #888888;">(3/13): python-keyczar-0.71c-2.el7.noarch.rpm | 218 kB 00:00:00 </span>
<span style="color: #888888;">warning: /var/cache/yum/x86_64/7Server/rhel-7-server-rpms/packages/python-babel-0.9.6-8.el7.noarch.rpm: Header V3 RSA/SHA256 Signature, key ID fd431d51: NOKEY</span>
<span style="color: #888888;">Public key for python-babel-0.9.6-8.el7.noarch.rpm is not installed</span>
<span style="color: #888888;">(4/13): python-babel-0.9.6-8.el7.noarch.rpm | 1.4 MB 00:00:02 </span>
<span style="color: #888888;">(5/13): python-markupsafe-0.11-10.el7.x86_64.rpm | 25 kB 00:00:00 </span>
<span style="color: #888888;">(6/13): python-httplib2-0.7.7-3.el7.noarch.rpm | 70 kB 00:00:02 </span>
<span style="color: #888888;">(7/13): python2-crypto-2.6.1-10.el7.x86_64.rpm | 475 kB 00:00:00 </span>
<span style="color: #888888;">Public key for python-jinja2-2.7.2-2.el7.noarch.rpm is not installed</span>
<span style="color: #888888;">(8/13): python-jinja2-2.7.2-2.el7.noarch.rpm | 516 kB 00:00:01 </span>
<span style="color: #888888;">(9/13): sshpass-1.05-5.el7.x86_64.rpm | 21 kB 00:00:00 </span>
<span style="color: #888888;">(10/13): python2-pyasn1-0.1.9-7.el7.noarch.rpm | 100 kB 00:00:00 </span>
<span style="color: #888888;">(11/13): python2-ecdsa-0.13-4.el7.noarch.rpm | 83 kB 00:00:01 </span>
<span style="color: #888888;">(12/13): python2-paramiko-1.16.1-1.el7.noarch.rpm | 257 kB 00:00:01 </span>
<span style="color: #888888;">(13/13): ansible-2.2.0.0-4.el7.noarch.rpm | 4.6 MB 00:00:09 </span>
<span style="color: #888888;">------------------------------------------------------------------------------------------------------------------------------------------------------</span>
<span style="color: #888888;">Total 846 kB/s | 7.9 MB 00:00:09 </span>
<span style="color: #888888;">Retrieving key from file:///etc/pki/rpm-gpg/RPM-GPG-KEY-redhat-release</span>
<span style="color: #888888;">Importing GPG key 0xFD431D51:</span>
<span style="color: #888888;"> Userid : "Red Hat, Inc. (release key 2) <security@redhat.com>"</span>
<span style="color: #888888;"> Fingerprint: 567e 347a d004 4ade 55ba 8a5f 199e 2f91 fd43 1d51</span>
<span style="color: #888888;"> Package : redhat-release-server-7.3-7.el7.x86_64 (installed)</span>
<span style="color: #888888;"> From : /etc/pki/rpm-gpg/RPM-GPG-KEY-redhat-release</span>
<span style="color: #888888;">Is this ok [y/N]: y</span>
<span style="color: #888888;">Importing GPG key 0x2FA658E0:</span>
<span style="color: #888888;"> Userid : "Red Hat, Inc. (auxiliary key) <security@redhat.com>"</span>
<span style="color: #888888;"> Fingerprint: 43a6 e49c 4a38 f4be 9abf 2a53 4568 9c88 2fa6 58e0</span>
<span style="color: #888888;"> Package : redhat-release-server-7.3-7.el7.x86_64 (installed)</span>
<span style="color: #888888;"> From : /etc/pki/rpm-gpg/RPM-GPG-KEY-redhat-release</span>
<span style="color: #888888;">Is this ok [y/N]: y</span>
<span style="color: #888888;">Running transaction check</span>
<span style="color: #888888;">Running transaction test</span>
<span style="color: #888888;">Transaction test succeeded</span>
<span style="color: #888888;">Running transaction</span>
<span style="color: #888888;"> Installing : sshpass-1.05-5.el7.x86_64 1/13 </span>
<span style="color: #888888;"> Installing : python-babel-0.9.6-8.el7.noarch 2/13 </span>
<span style="color: #888888;"> Installing : python2-pyasn1-0.1.9-7.el7.noarch 3/13 </span>
<span style="color: #888888;"> Installing : libtommath-0.42.0-4.el7.x86_64 4/13 </span>
<span style="color: #888888;"> Installing : libtomcrypt-1.17-23.el7.x86_64 5/13 </span>
<span style="color: #888888;"> Installing : python2-crypto-2.6.1-10.el7.x86_64 6/13 </span>
<span style="color: #888888;"> Installing : python-keyczar-0.71c-2.el7.noarch 7/13 </span>
<span style="color: #888888;"> Installing : python2-ecdsa-0.13-4.el7.noarch 8/13 </span>
<span style="color: #888888;"> Installing : python2-paramiko-1.16.1-1.el7.noarch 9/13 </span>
<span style="color: #888888;"> Installing : python-httplib2-0.7.7-3.el7.noarch 10/13 </span>
<span style="color: #888888;"> Installing : python-markupsafe-0.11-10.el7.x86_64 11/13 </span>
<span style="color: #888888;"> Installing : python-jinja2-2.7.2-2.el7.noarch 12/13 </span>
<span style="color: #888888;"> Installing : ansible-2.2.0.0-4.el7.noarch 13/13 </span>
<span style="color: #888888;"> Verifying : python-keyczar-0.71c-2.el7.noarch 1/13 </span>
<span style="color: #888888;"> Verifying : python-markupsafe-0.11-10.el7.x86_64 2/13 </span>
<span style="color: #888888;"> Verifying : python-jinja2-2.7.2-2.el7.noarch 3/13 </span>
<span style="color: #888888;"> Verifying : python-httplib2-0.7.7-3.el7.noarch 4/13 </span>
<span style="color: #888888;"> Verifying : python2-ecdsa-0.13-4.el7.noarch 5/13 </span>
<span style="color: #888888;"> Verifying : libtomcrypt-1.17-23.el7.x86_64 6/13 </span>
<span style="color: #888888;"> Verifying : ansible-2.2.0.0-4.el7.noarch 7/13 </span>
<span style="color: #888888;"> Verifying : libtommath-0.42.0-4.el7.x86_64 8/13 </span>
<span style="color: #888888;"> Verifying : python2-pyasn1-0.1.9-7.el7.noarch 9/13 </span>
<span style="color: #888888;"> Verifying : python-babel-0.9.6-8.el7.noarch 10/13 </span>
<span style="color: #888888;"> Verifying : python2-crypto-2.6.1-10.el7.x86_64 11/13 </span>
<span style="color: #888888;"> Verifying : python2-paramiko-1.16.1-1.el7.noarch 12/13 </span>
<span style="color: #888888;"> Verifying : sshpass-1.05-5.el7.x86_64 13/13 </span>
<span style="color: #888888;">Installed:</span>
<span style="color: #888888;"> ansible.noarch 0:2.2.0.0-4.el7 </span>
<span style="color: #888888;">Dependency Installed:</span>
<span style="color: #888888;"> libtomcrypt.x86_64 0:1.17-23.el7 libtommath.x86_64 0:0.42.0-4.el7 python-babel.noarch 0:0.9.6-8.el7 </span>
<span style="color: #888888;"> python-httplib2.noarch 0:0.7.7-3.el7 python-jinja2.noarch 0:2.7.2-2.el7 python-keyczar.noarch 0:0.71c-2.el7 </span>
<span style="color: #888888;"> python-markupsafe.x86_64 0:0.11-10.el7 python2-crypto.x86_64 0:2.6.1-10.el7 python2-ecdsa.noarch 0:0.13-4.el7 </span>
<span style="color: #888888;"> python2-paramiko.noarch 0:1.16.1-1.el7 python2-pyasn1.noarch 0:0.1.9-7.el7 sshpass.x86_64 0:1.05-5.el7 </span>
<span style="color: #888888;">Complete!</span>
</pre>
</div>
<br />
<br />
<b>/var</b> sizing by requirements must more than 10Gb<br />
If you download RHEL qcow2 image from Redhat Portal, a /var filesystem is in / root filesystem that space less than Ansible Tower recommentation. So you should create new disk and mount as /var instead of existing /var in / root filesystem.<br />
<br />
<!-- HTML generated using hilite.me --><br />
<div style="background: #f8f8f8; border-width: 0.1em 0.1em 0.1em 0.8em; border: solid gray; overflow: auto; padding: 0.2em 0.6em; width: auto;">
<pre style="line-height: 125%; margin: 0;"><span style="color: #888888;">$ sudo mkdir /tmp/var</span>
<span style="color: #888888;">$ sudo mount /dev/vdb /tmp/var/</span>
<span style="color: #888888;">$ sudo rsync -avz /var/ /tmp/var/</span>
<span style="color: #888888;">$ sudo blkid</span></pre>
<pre style="line-height: 125%; margin: 0;"></pre>
<pre style="line-height: 125%; margin: 0;">Add new line
<span style="color: #888888;">$ sudo vi /etc/fstab</span></pre>
<pre style="line-height: 125%; margin: 0;"><span style="color: #888888;">UUID=ac1cda02-5a44-4b04-b6ad-a28bf69639e6 /var xfs defaults 0 2</span>
</pre>
<pre style="line-height: 125%; margin: 0;">Reboot<span style="color: #888888;"> </span>
<span style="color: #888888;">$ sudo restorecon -R /var</span>
</pre>
</div>
<br />
<br />
<b>Starting, Stopping, and Restarting Tower</b><br />
<br />
/usr/bin/ansible-tower-service [ start | stop | restart | status ]<br />
<br />
<!-- HTML generated using hilite.me --><br />
<div style="background: #f8f8f8; border-width: 0.1em 0.1em 0.1em 0.8em; border: solid gray; overflow: auto; padding: 0.2em 0.6em; width: auto;">
<pre style="line-height: 125%; margin: 0;"><span style="color: navy; font-weight: bold;">[jboss@rhelnode1 ~]$</span> ansible-tower-service status
<span style="color: #888888;">Showing Tower Status</span>
<span style="color: #888888;">Redirecting to /bin/systemctl status postgresql-9.4.service</span>
<span style="color: lime;">●</span><span style="color: #888888;"> postgresql-9.4.service - PostgreSQL 9.4 database server</span>
<span style="color: #888888;"> Loaded: loaded (/usr/lib/systemd/system/postgresql-9.4.service; enabled; vendor preset: disabled)</span>
<span style="color: #888888;"> Active: </span><span style="color: lime;">active (running)</span><span style="color: #888888;"> since อ. 2017-01-10 21:30:22 ICT; 20min ago</span>
<span style="color: #888888;"> Process: 890 ExecStart=/usr/pgsql-9.4/bin/pg_ctl start -D ${PGDATA} -s -w -t 300 (code=exited, status=0/SUCCESS)</span>
<span style="color: #888888;"> Process: 845 ExecStartPre=/usr/pgsql-9.4/bin/postgresql94-check-db-dir ${PGDATA} (code=exited, status=0/SUCCESS)</span>
<span style="color: #888888;"> Main PID: 1990 (postgres)</span>
<span style="color: #888888;"> CGroup: /system.slice/postgresql-9.4.service</span>
<span style="color: #888888;"> ├─1990 /usr/pgsql-9.4/bin/postgres -D /var/lib/pgsql/9.4/data</span>
<span style="color: #888888;"> ├─2167 postgres: logger process </span>
<span style="color: #888888;"> ├─2202 postgres: checkpointer process </span>
<span style="color: #888888;"> ├─2203 postgres: writer process </span>
<span style="color: #888888;"> ├─2204 postgres: wal writer process </span>
<span style="color: #888888;"> ├─2205 postgres: autovacuum launcher process </span>
<span style="color: #888888;"> ├─2206 postgres: stats collector process </span>
<span style="color: #888888;"> ├─2249 postgres: awx awx 127.0.0.1(60506) idle</span>
<span style="color: #888888;"> ├─2250 postgres: awx awx 127.0.0.1(60508) idle</span>
<span style="color: #888888;"> ├─2251 postgres: awx awx 127.0.0.1(60510) idle</span>
<span style="color: #888888;"> └─2253 postgres: awx awx 127.0.0.1(60530) idle</span>
<span style="color: #888888;">Redirecting to /bin/systemctl status redis.service</span>
<span style="color: lime;">●</span><span style="color: #888888;"> redis.service - Redis persistent key-value database</span>
<span style="color: #888888;"> Loaded: loaded (/usr/lib/systemd/system/redis.service; enabled; vendor preset: disabled)</span>
<span style="color: #888888;"> Drop-In: /etc/systemd/system/redis.service.d</span>
<span style="color: #888888;"> └─limit.conf</span>
<span style="color: #888888;"> Active: </span><span style="color: lime;">active (running)</span><span style="color: #888888;"> since อ. 2017-01-10 21:30:06 ICT; 20min ago</span>
<span style="color: #888888;"> Main PID: 841 (redis-server)</span>
<span style="color: #888888;"> CGroup: /system.slice/redis.service</span>
<span style="color: #888888;"> └─841 /usr/bin/redis-server 127.0.0.1:6379</span>
<span style="color: #888888;">Redirecting to /bin/systemctl status httpd.service</span>
<span style="color: lime;">●</span><span style="color: #888888;"> httpd.service - The Apache HTTP Server</span>
<span style="color: #888888;"> Loaded: loaded (/usr/lib/systemd/system/httpd.service; enabled; vendor preset: disabled)</span>
<span style="color: #888888;"> Active: </span><span style="color: lime;">active (running)</span><span style="color: #888888;"> since อ. 2017-01-10 21:30:18 ICT; 20min ago</span>
<span style="color: #888888;"> Docs: man:httpd(8)</span>
<span style="color: #888888;"> man:apachectl(8)</span>
<span style="color: #888888;"> Main PID: 853 (httpd)</span>
<span style="color: #888888;"> Status: "Total requests: 59; Current requests/sec: 0; Current traffic: 0 B/sec"</span>
<span style="color: #888888;"> CGroup: /system.slice/httpd.service</span>
<span style="color: #888888;"> ├─ 853 /usr/sbin/httpd -DFOREGROUND</span>
<span style="color: #888888;"> ├─2070 (wsgi:awx) -DFOREGROUND</span>
<span style="color: #888888;"> ├─2071 (wsgi:awx) -DFOREGROUND</span>
<span style="color: #888888;"> ├─2072 (wsgi:awx_plain -DFOREGROUND</span>
<span style="color: #888888;"> ├─2073 (wsgi:awx_plain -DFOREGROUND</span>
<span style="color: #888888;"> ├─2074 /usr/sbin/httpd -DFOREGROUND</span>
<span style="color: #888888;"> ├─2075 /usr/sbin/httpd -DFOREGROUND</span>
<span style="color: #888888;"> ├─2076 /usr/sbin/httpd -DFOREGROUND</span>
<span style="color: #888888;"> ├─2077 /usr/sbin/httpd -DFOREGROUND</span>
<span style="color: #888888;"> ├─2078 /usr/sbin/httpd -DFOREGROUND</span>
<span style="color: #888888;"> ├─2302 /usr/sbin/httpd -DFOREGROUND</span>
<span style="color: #888888;"> ├─2307 /usr/sbin/httpd -DFOREGROUND</span>
<span style="color: #888888;"> ├─2308 /usr/sbin/httpd -DFOREGROUND</span>
<span style="color: #888888;"> ├─2310 /usr/sbin/httpd -DFOREGROUND</span>
<span style="color: #888888;"> └─2329 /usr/sbin/httpd -DFOREGROUND</span>
<span style="color: #888888;">Redirecting to /bin/systemctl status supervisord.service</span>
<span style="color: lime;">●</span><span style="color: #888888;"> supervisord.service - Process Monitoring and Control Daemon</span>
<span style="color: #888888;"> Loaded: loaded (/usr/lib/systemd/system/supervisord.service; enabled; vendor preset: disabled)</span>
<span style="color: #888888;"> Active: </span><span style="color: lime;">active (running)</span><span style="color: #888888;"> since อ. 2017-01-10 21:30:03 ICT; 20min ago</span>
<span style="color: #888888;"> Process: 510 ExecStart=/usr/bin/supervisord -c /etc/supervisord.conf (code=exited, status=0/SUCCESS)</span>
<span style="color: #888888;"> Main PID: 642 (supervisord)</span>
<span style="color: #888888;"> CGroup: /system.slice/supervisord.service</span>
<span style="color: #888888;"> ├─ 642 /usr/bin/python /usr/bin/supervisord -c /etc/supervisord.conf</span>
<span style="color: #888888;"> ├─ 659 python /usr/bin/tower-manage run_socketio_service</span>
<span style="color: #888888;"> ├─ 660 python /usr/bin/tower-manage run_fact_cache_receiver</span>
<span style="color: #888888;"> ├─ 661 python /usr/bin/tower-manage run_task_system</span>
<span style="color: #888888;"> ├─ 662 python /usr/bin/tower-manage celery worker -B -l debug --autoscale=50,4 -Ofair -s /var/lib/awx/beat.db</span>
<span style="color: #888888;"> ├─ 663 python /usr/bin/tower-manage run_callback_receiver</span>
<span style="color: #888888;"> ├─2256 python /usr/bin/tower-manage run_callback_receiver</span>
<span style="color: #888888;"> ├─2257 python /usr/bin/tower-manage run_callback_receiver</span>
<span style="color: #888888;"> ├─2258 python /usr/bin/tower-manage run_callback_receiver</span>
<span style="color: #888888;"> ├─2259 python /usr/bin/tower-manage run_callback_receiver</span>
<span style="color: #888888;"> ├─2260 python /usr/bin/tower-manage run_callback_receiver</span>
<span style="color: #888888;"> ├─2279 python /usr/bin/tower-manage celery worker -B -l debug --autoscale=50,4 -Ofair -s /var/lib/awx/beat.db</span>
<span style="color: #888888;"> ├─2280 python /usr/bin/tower-manage celery worker -B -l debug --autoscale=50,4 -Ofair -s /var/lib/awx/beat.db</span>
<span style="color: #888888;"> ├─2281 python /usr/bin/tower-manage celery worker -B -l debug --autoscale=50,4 -Ofair -s /var/lib/awx/beat.db</span>
<span style="color: #888888;"> ├─2282 python /usr/bin/tower-manage celery worker -B -l debug --autoscale=50,4 -Ofair -s /var/lib/awx/beat.db</span>
<span style="color: #888888;"> └─2283 python /usr/bin/tower-manage celery worker -B -l debug --autoscale=50,4 -Ofair -s /var/lib/awx/beat.db</span>
</pre>
</div>
<br />
<br />
Change admin user password<br />
<br />
<!-- HTML generated using hilite.me --><br />
<div style="background: #f8f8f8; border-width: 0.1em 0.1em 0.1em 0.8em; border: solid gray; overflow: auto; padding: 0.2em 0.6em; width: auto;">
<pre style="line-height: 125%; margin: 0;"><span style="color: navy; font-weight: bold;">[jboss@rhelnode1 ~]$</span> sudo tower-manage changepassword admin
<span style="color: #888888;">[sudo] password for jboss: </span>
<span style="color: #888888;">Changing password for user 'admin'</span>
<span style="color: #888888;">Password: </span>
<span style="color: #888888;">Password (again): </span>
<span style="color: #888888;">Password changed successfully for user 'admin'</span>
</pre>
</div>
<br />
<div class="separator" style="clear: both; text-align: center;">
</div>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhIUbGQ1F92iIkLVi7bAvkImzhmdb6RRaKgW3f1ivGaj-xjv1qcan69ffz0BR4IxF_fKy3L96M-siSVnslt4NaPTPW-SRr1n5xAgSGG3uqk0oNOZwZSl95hweudAkyY_fb_ICwD4pLC6rwq/s1600/Selection_006.jpg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="457" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhIUbGQ1F92iIkLVi7bAvkImzhmdb6RRaKgW3f1ivGaj-xjv1qcan69ffz0BR4IxF_fKy3L96M-siSVnslt4NaPTPW-SRr1n5xAgSGG3uqk0oNOZwZSl95hweudAkyY_fb_ICwD4pLC6rwq/s640/Selection_006.jpg" width="640" /></a></div>
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />พี่นัทhttp://www.blogger.com/profile/14291406590594822628noreply@blogger.com0tag:blogger.com,1999:blog-2488490746136830746.post-29556259501357127192017-01-10T10:50:00.001+07:002017-01-10T10:50:21.748+07:00Changing the password on the RHEL 7 kvm qcow2<br />
<br />
<div style="background: #f8f8f8; border-width: 0.1em 0.1em 0.1em 0.8em; border: solid gray; overflow: auto; padding: 0.2em 0.6em; width: auto;">
<pre style="line-height: 125%; margin: 0;"><span style="color: navy; font-weight: bold;">[nutt@nutt-pc ~]$</span> sudo apt-get install libguestfs-tools
<span style="color: #888888;">[sudo] password for nutt: </span>
<span style="color: #888888;">Sorry, try again.</span>
<span style="color: #888888;">[sudo] password for nutt: </span>
<span style="color: #888888;">Reading package lists... Done</span>
<span style="color: #888888;">Building dependency tree </span>
<span style="color: #888888;">Reading state information... Done</span>
<span style="color: #888888;">The following packages were automatically installed and are no longer required:</span>
<span style="color: #888888;"> archdetect-deb authbind curtin-common dbconfig-common dbconfig-pgsql</span>
<span style="color: #888888;"> distro-info docutils-common fonts-font-awesome formencode-i18n</span>
<span style="color: #888888;"> freeipmi-common freeipmi-tools ieee-data isc-dhcp-server</span>
<span style="color: #888888;"> libconfig-general-perl libdebian-installer4 libecap3 libfreeipmi16</span>
<span style="color: #888888;"> libibverbs1 libipmiconsole2 libipmidetect0 libirs-export141</span>
<span style="color: #888888;"> libisccfg-export140 libjs-angularjs libjs-modernizr libjs-sphinxdoc</span>
<span style="color: #888888;"> libjs-yui3-common libjs-yui3-full libjs-yui3-min librdmacm1</span>
<span style="color: #888888;"> linux-headers-4.4.0-21 linux-headers-4.4.0-21-generic linux-headers-4.4.0-28</span>
<span style="color: #888888;"> linux-headers-4.4.0-28-generic linux-headers-4.4.0-31</span>
<span style="color: #888888;"> linux-headers-4.4.0-31-generic linux-headers-4.4.0-34</span>
<span style="color: #888888;"> linux-headers-4.4.0-34-generic linux-headers-4.4.0-38</span>
<span style="color: #888888;"> linux-headers-4.4.0-38-generic linux-headers-4.4.0-42</span>
<span style="color: #888888;"> linux-headers-4.4.0-42-generic linux-headers-4.4.0-43</span>
<span style="color: #888888;"> linux-headers-4.4.0-43-generic linux-headers-4.4.0-45</span>
<span style="color: #888888;"> linux-headers-4.4.0-45-generic linux-headers-4.4.0-47</span>
<span style="color: #888888;"> linux-headers-4.4.0-47-generic linux-image-4.4.0-43-generic</span>
<span style="color: #888888;"> linux-image-4.4.0-45-generic linux-image-4.4.0-47-generic</span>
<span style="color: #888888;"> linux-image-extra-4.4.0-43-generic linux-image-extra-4.4.0-45-generic</span>
<span style="color: #888888;"> linux-image-extra-4.4.0-47-generic postgresql pxelinux</span>
<span style="color: #888888;"> python-babel-localedata python-django-common python3-alabaster python3-attr</span>
<span style="color: #888888;"> python3-babel python3-bson python3-bson-ext python3-convoy python3-crochet</span>
<span style="color: #888888;"> python3-curtin python3-distro-info python3-django python3-django-piston3</span>
<span style="color: #888888;"> python3-djorm-ext-pgarray python3-dnspython python3-docutils python3-ecdsa</span>
<span style="color: #888888;"> python3-formencode python3-iso8601 python3-jsonschema python3-mimeparse</span>
<span style="color: #888888;"> python3-netaddr python3-netifaces python3-oauth python3-pam python3-paramiko</span>
<span style="color: #888888;"> python3-pbr python3-petname python3-prettytable python3-psycopg2</span>
<span style="color: #888888;"> python3-pyasn1-modules python3-pygments python3-pyvmomi python3-roman</span>
<span style="color: #888888;"> python3-seamicroclient python3-serial python3-service-identity</span>
<span style="color: #888888;"> python3-simplejson python3-simplestreams python3-sphinx</span>
<span style="color: #888888;"> python3-sphinx-rtd-theme python3-sqlparse python3-tempita python3-twisted</span>
<span style="color: #888888;"> python3-txtftp python3-tz python3-yaml python3-zope.interface sphinx-common</span>
<span style="color: #888888;"> sphinx-rtd-theme-common squid squid-common squid-langpack tgt</span>
<span style="color: #888888;"> ubuntu-cloudimage-keyring</span>
<span style="color: #888888;">Use 'sudo apt autoremove' to remove them.</span>
<span style="color: #888888;">The following additional packages will be installed:</span>
<span style="color: #888888;"> btrfs-tools cryptsetup cryptsetup-bin dmeventd dmsetup extlinux hfsplus</span>
<span style="color: #888888;"> libconfig9 libdevmapper-event1.02.1 libguestfs-hfsplus libguestfs-perl</span>
<span style="color: #888888;"> libguestfs-reiserfs libguestfs-xfs libguestfs0 libhfsp0 libhivex0</span>
<span style="color: #888888;"> liblvm2app2.2 liblvm2cmd2.02 libreadline5 libstring-shellquote-perl</span>
<span style="color: #888888;"> libsys-virt-perl libwin-hivex-perl libxml-xpath-perl lsscsi lvm2 lzop mdadm</span>
<span style="color: #888888;"> reiserfsprogs scrub supermin xfsprogs zerofree</span>
<span style="color: #888888;">Suggested packages:</span>
<span style="color: #888888;"> keyutils libguestfs-gfs2 libguestfs-jfs libguestfs-nilfs libguestfs-rescue</span>
<span style="color: #888888;"> libguestfs-rsync libguestfs-zfs thin-provisioning-tools default-mta</span>
<span style="color: #888888;"> | mail-transport-agent xfsdump quota</span>
<span style="color: #888888;">The following NEW packages will be installed:</span>
<span style="color: #888888;"> btrfs-tools cryptsetup cryptsetup-bin dmeventd dmsetup extlinux hfsplus</span>
<span style="color: #888888;"> libconfig9 libdevmapper-event1.02.1 libguestfs-hfsplus libguestfs-perl</span>
<span style="color: #888888;"> libguestfs-reiserfs libguestfs-tools libguestfs-xfs libguestfs0 libhfsp0</span>
<span style="color: #888888;"> libhivex0 liblvm2app2.2 liblvm2cmd2.02 libreadline5</span>
<span style="color: #888888;"> libstring-shellquote-perl libsys-virt-perl libwin-hivex-perl</span>
<span style="color: #888888;"> libxml-xpath-perl lsscsi lvm2 lzop mdadm reiserfsprogs scrub supermin</span>
<span style="color: #888888;"> xfsprogs zerofree</span>
<span style="color: #888888;">0 upgraded, 33 newly installed, 0 to remove and 14 not upgraded.</span>
<span style="color: #888888;">Need to get 10.5 MB of archives.</span>
<span style="color: #888888;">After this operation, 48.2 MB of additional disk space will be used.</span>
<span style="color: #888888;">Do you want to continue? [Y/n] Y</span>
</pre>
</div>
<br />
<!-- HTML generated using hilite.me --><br />
Generate encrypted password with openssl then copy output string and paste in shadow file in next steps<br />
<br />
<!-- HTML generated using hilite.me --><br />
<div style="background: #f8f8f8; border-width: 0.1em 0.1em 0.1em 0.8em; border: solid gray; overflow: auto; padding: 0.2em 0.6em; width: auto;">
<pre style="line-height: 125%; margin: 0;"><span style="color: navy; font-weight: bold;">[nutt@nutt-pc KVM]$</span> openssl passwd -1 changeme
<span style="color: #888888;">$1$twJh0TGA$ebVI529VkTMaICf0Oyvac/</span>
</pre>
</div>
<br />
<br />
<div style="background: #f8f8f8; border-width: 0.1em 0.1em 0.1em 0.8em; border: solid gray; overflow: auto; padding: 0.2em 0.6em; width: auto;">
<pre style="line-height: 125%; margin: 0;"><span style="color: navy; font-weight: bold;">[nutt@nutt-pc KVM]$</span> sudo guestfish --rw -a ./RHEL7_Node2.qcow2
<span style="color: #888888;">Welcome to guestfish, the guest filesystem shell for</span>
<span style="color: #888888;">editing virtual machine filesystems and disk images.</span>
<span style="color: #888888;">Type: 'help' for help on commands</span>
<span style="color: #888888;"> 'man' to read the manual</span>
<span style="color: #888888;"> 'quit' to quit the shell</span>
<span style="color: #888888;">><fs> run</span>
<span style="color: #888888;"> 100% ⟦▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒▒⟧ 00:00</span>
<span style="color: #888888;">><fs> list-filesystems</span>
<span style="color: #888888;">/dev/sda1: xfs</span>
<span style="color: #888888;">><fs> mount /dev/sda1 /</span>
<span style="color: #888888;">><fs> df</span>
<span style="color: #888888;">Filesystem 1K-blocks Used Available Use% Mounted on</span>
<span style="color: #888888;">/dev/root 4128448 424048 3494688 11% /</span>
<span style="color: #888888;">tmpfs 97548 292 97256 1% /run</span>
<span style="color: #888888;">/dev 240748 0 240748 0% /dev</span>
<span style="color: #888888;">/dev/sda1 8181760 970692 7211068 12% /sysroot</span>
<span style="color: #888888;">><fs> vi /etc/shadow</span>
<span style="color: #888888;">><fs> quit</span>
</pre>
</div>
<br />
<br />
<br />
Reference: <a href="https://access.redhat.com/discussions/664843">https://access.redhat.com/discussions/664843</a><br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />พี่นัทhttp://www.blogger.com/profile/14291406590594822628noreply@blogger.com0tag:blogger.com,1999:blog-2488490746136830746.post-52291823126026119512017-01-05T16:19:00.001+07:002017-01-05T16:19:20.336+07:00MongoDB : Enable Auth<br />
<br />
<br />
When authentication must specific database which user are.<br />
When creating new user must specific database that user would like to have a specific provileges.<br />
<br />
<!-- HTML generated using hilite.me --><br />
<div style="background: #f8f8f8; border-width: 0.1em 0.1em 0.1em 0.8em; border: solid gray; overflow: auto; padding: 0.2em 0.6em; width: auto;">
<pre style="line-height: 125%; margin: 0;"><span style="color: navy; font-weight: bold;">></span> use admin
<span style="color: #888888;">switched to db admin</span>
<span style="color: navy; font-weight: bold;">></span> db.auth<span style="color: #666666;">(</span><span style="color: #ba2121;">"myUserAdmin"</span>, <span style="color: #ba2121;">"P@ssw0rd1"</span><span style="color: #666666;">)</span>
<span style="color: #888888;">1</span>
<span style="color: navy; font-weight: bold;">></span> use settrade
<span style="color: #888888;">switched to db settrade</span>
<span style="color: navy; font-weight: bold;">></span> db.createUser<span style="color: #666666;">(</span>
<span style="color: #888888;">... {</span>
<span style="color: #888888;">... user: "mayfly",</span>
<span style="color: #888888;">... pwd: "P@ssw0rd1",</span>
<span style="color: #888888;">... roles: [ { role: "readWrite", db: "settrade" },</span>
<span style="color: #888888;">... { role: "read", db: "pantip" } ]</span>
<span style="color: #888888;">... }</span>
<span style="color: #888888;">... )</span>
<span style="color: #888888;">Successfully added user: {</span>
<span style="color: #888888;"> "user" : "mayfly",</span>
<span style="color: #888888;"> "roles" : [</span>
<span style="color: #888888;"> {</span>
<span style="color: #888888;"> "role" : "readWrite",</span>
<span style="color: #888888;"> "db" : "settrade"</span>
<span style="color: #888888;"> },</span>
<span style="color: #888888;"> {</span>
<span style="color: #888888;"> "role" : "read",</span>
<span style="color: #888888;"> "db" : "pantip"</span>
<span style="color: #888888;"> }</span>
<span style="color: #888888;"> ]</span>
<span style="color: #888888;">}</span>
<span style="color: navy; font-weight: bold;">></span> db.system.users.find<span style="color: #666666;">()</span>
<span style="color: #888888;">{ "_id" : ObjectId("586db42d5b10326487fc702e"), "user" : "mayfly", "pwd" : "04acf20f8062345c503b5296ce094b59", "roles" : [ "readWrite" ] }</span>
<span style="color: #888888;">{ "_id" : ObjectId("586db82b5b10326487fc70ab"), "user" : "termitequeen", "pwd" : "7027db6319fd71eba513904a3f9aff4e", "roles" : [ "dbAdmin" ] }</span>
<span style="color: navy; font-weight: bold;">></span> use admin
<span style="color: #888888;">switched to db admin</span>
<span style="color: navy; font-weight: bold;">></span> db.system.users.find<span style="color: #666666;">()</span>
<span style="color: #888888;">{ "_id" : "admin.myUserAdmin", "user" : "myUserAdmin", "db" : "admin", "credentials" : { "SCRAM-SHA-1" : { "iterationCount" : 10000, "salt" : "/Il1MVz+E6PswLoN1RNBGw==", "storedKey" : "njrNzk1dnB5fjQTzmwZgVZ3NtBs=", "serverKey" : "pOgAZRsLN1Gr82y4Y7ZElshExlw=" } }, "roles" : [ { "role" : "userAdminAnyDatabase", "db" : "admin" } ] }</span>
<span style="color: #888888;">{ "_id" : "settrade.queen", "user" : "queen", "db" : "settrade", "credentials" : { "SCRAM-SHA-1" : { "iterationCount" : 10000, "salt" : "NWY5bEymoPHiDxt9tFFrgA==", "storedKey" : "iLUtPZvZoX1PAYOek2xPmMN8npE=", "serverKey" : "fY25R+QqcyiRrCh6C0HbOOqAFSQ=" } }, "roles" : [ { "role" : "userAdmin", "db" : "settrade" }, { "role" : "dbOwner", "db" : "settrade" } ] }</span>
<span style="color: #888888;">{ "_id" : "settrade.tester", "user" : "tester", "db" : "settrade", "credentials" : { "SCRAM-SHA-1" : { "iterationCount" : 10000, "salt" : "rhNPFiBo1HPPFV3yveruSA==", "storedKey" : "OYw6GgZE/oXTgw1PTeGC2CD80io=", "serverKey" : "sPtXbzs4lzpp0+nLOa9dAuBCVRw=" } }, "roles" : [ { "role" : "readWrite", "db" : "settrade" }, { "role" : "read", "db" : "pantip" } ] }</span>
<span style="color: #888888;">{ "_id" : "settrade.mayfly", "user" : "mayfly", "db" : "settrade", "credentials" : { "SCRAM-SHA-1" : { "iterationCount" : 10000, "salt" : "j6uKBrG9tWOI/Tr09MM9zA==", "storedKey" : "VIntdHaC91hTWytj071wrI6T/pI=", "serverKey" : "bhitcPVJSz4VsIxZcC5LvP2tS4o=" } }, "roles" : [ { "role" : "readWrite", "db" : "settrade" }, { "role" : "read", "db" : "pantip" } ] }</span>
<span style="color: navy; font-weight: bold;">></span> db.createUser<span style="color: #666666;">(</span>
<span style="color: #888888;">... {</span>
<span style="color: #888888;">... user: "nutt",</span>
<span style="color: #888888;">... pwd: "P@ssw0rd1",</span>
<span style="color: #888888;">... roles: [ { role: "root", db: "admin" } ]</span>
<span style="color: #888888;">... }</span>
<span style="color: #888888;">... )</span>
<span style="color: #888888;">Successfully added user: {</span>
<span style="color: #888888;"> "user" : "nutt",</span>
<span style="color: #888888;"> "roles" : [</span>
<span style="color: #888888;"> {</span>
<span style="color: #888888;"> "role" : "root",</span>
<span style="color: #888888;"> "db" : "admin"</span>
<span style="color: #888888;"> }</span>
<span style="color: #888888;"> ]</span>
<span style="color: #888888;">}</span>
<span style="color: navy; font-weight: bold;">></span> db.system.users.find<span style="color: #666666;">()</span>
<span style="color: #888888;">{ "_id" : "admin.myUserAdmin", "user" : "myUserAdmin", "db" : "admin", "credentials" : { "SCRAM-SHA-1" : { "iterationCount" : 10000, "salt" : "/Il1MVz+E6PswLoN1RNBGw==", "storedKey" : "njrNzk1dnB5fjQTzmwZgVZ3NtBs=", "serverKey" : "pOgAZRsLN1Gr82y4Y7ZElshExlw=" } }, "roles" : [ { "role" : "userAdminAnyDatabase", "db" : "admin" } ] }</span>
<span style="color: #888888;">{ "_id" : "settrade.queen", "user" : "queen", "db" : "settrade", "credentials" : { "SCRAM-SHA-1" : { "iterationCount" : 10000, "salt" : "NWY5bEymoPHiDxt9tFFrgA==", "storedKey" : "iLUtPZvZoX1PAYOek2xPmMN8npE=", "serverKey" : "fY25R+QqcyiRrCh6C0HbOOqAFSQ=" } }, "roles" : [ { "role" : "userAdmin", "db" : "settrade" }, { "role" : "dbOwner", "db" : "settrade" } ] }</span>
<span style="color: #888888;">{ "_id" : "settrade.tester", "user" : "tester", "db" : "settrade", "credentials" : { "SCRAM-SHA-1" : { "iterationCount" : 10000, "salt" : "rhNPFiBo1HPPFV3yveruSA==", "storedKey" : "OYw6GgZE/oXTgw1PTeGC2CD80io=", "serverKey" : "sPtXbzs4lzpp0+nLOa9dAuBCVRw=" } }, "roles" : [ { "role" : "readWrite", "db" : "settrade" }, { "role" : "read", "db" : "pantip" } ] }</span>
<span style="color: #888888;">{ "_id" : "settrade.mayfly", "user" : "mayfly", "db" : "settrade", "credentials" : { "SCRAM-SHA-1" : { "iterationCount" : 10000, "salt" : "j6uKBrG9tWOI/Tr09MM9zA==", "storedKey" : "VIntdHaC91hTWytj071wrI6T/pI=", "serverKey" : "bhitcPVJSz4VsIxZcC5LvP2tS4o=" } }, "roles" : [ { "role" : "readWrite", "db" : "settrade" }, { "role" : "read", "db" : "pantip" } ] }</span>
<span style="color: #888888;">{ "_id" : "admin.nutt", "user" : "nutt", "db" : "admin", "credentials" : { "SCRAM-SHA-1" : { "iterationCount" : 10000, "salt" : "dOXFYnX4lqt9+sn3Tel5eg==", "storedKey" : "wIsiKjB324I+lzac3/Dio+kmFe0=", "serverKey" : "2Z/g2ScIF6N46WcpjO88fzX57FY=" } }, "roles" : [ { "role" : "root", "db" : "admin" } ] }</span>
</pre>
</div>
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />พี่นัทhttp://www.blogger.com/profile/14291406590594822628noreply@blogger.com0tag:blogger.com,1999:blog-2488490746136830746.post-1913940338440644582016-12-30T22:18:00.001+07:002016-12-30T22:21:22.605+07:00Docker Daemon<br />
<br />
The Docker daemon can listen for Docker Remote API requests via three different types of Socket: <b>unix</b>, <b>tcp</b>, and <b>fd</b><br />
<br />
<b>unix</b> :- /var/run/docker.sock (default)<br />
<br />
<b>tcp</b> :- for remote access, support tls1.0 i.e.<br />
-H tcp://0.0.0.0:2375 # for all IP address<br />
-H tcp://192.168.59.103:2375 # specific IP address<br />
*** Docker daemon: 2375 (non-secure), 2376 (<a href="https://docs.docker.com/engine/security/https/" target="_blank">secure</a>)<br />
<br />
<b>fd</b> :- socket activated files i.e.<br />
-H fd://<br />
-H fd://3<br />
<br />
Can specific multiple socket like this<br />
dockerd -H <b>unix</b>:///var/run/docker.sock -H <b>tcp</b>://192.168.59.106 -H <b>tcp</b>://10.10.10.2<br />
<br />
<br />พี่นัทhttp://www.blogger.com/profile/14291406590594822628noreply@blogger.com0tag:blogger.com,1999:blog-2488490746136830746.post-33001425864175109952016-12-22T14:17:00.000+07:002016-12-22T15:18:46.369+07:00How to deploy scrapy project to scrapinghub.com then save crawling data to mongoDB<br />
<br />
<br />
PIP required module :<br />
pymongo<br />
<!-- HTML generated using hilite.me --><br />
<div style="background: #f8f8f8; border-width: 0.1em 0.1em 0.1em 0.8em; border: solid gray; overflow: auto; padding: 0.2em 0.6em; width: auto;">
<pre style="line-height: 125%; margin: 0;"><span style="color: navy; font-weight: bold;">ubuntu@nutthaphongmail:~$</span> docker <span style="color: green;">exec</span> -it --user root scrapyd-server bash
<span style="color: navy; font-weight: bold;">root@6b82490da131:/home/scrapyd#</span> pip install shub
<span style="color: #888888;">Collecting shub</span>
<span style="color: #888888;"> Downloading shub-2.5.0-py2.py3-none-any.whl (47kB)</span>
<span style="color: #888888;"> 100% |################################| 51kB 773kB/s </span>
<span style="color: #888888;">Collecting requests (from shub)</span>
<span style="color: #888888;"> Downloading requests-2.12.4-py2.py3-none-any.whl (576kB)</span>
<span style="color: #888888;"> 100% |################################| 583kB 1.3MB/s </span>
<span style="color: #888888;">Requirement already satisfied: six in /usr/local/lib/python2.7/dist-packages (from shub)</span>
<span style="color: #888888;">Collecting scrapinghub (from shub)</span>
<span style="color: #888888;"> Downloading scrapinghub-1.9.0-py2-none-any.whl</span>
<span style="color: #888888;">Requirement already satisfied: pip in /usr/local/lib/python2.7/dist-packages (from shub)</span>
<span style="color: #888888;">Collecting PyYAML (from shub)</span>
<span style="color: #888888;"> Downloading PyYAML-3.12.tar.gz (253kB)</span>
<span style="color: #888888;"> 100% |################################| 256kB 2.5MB/s </span>
<span style="color: #888888;">Collecting click (from shub)</span>
<span style="color: #888888;"> Downloading click-6.6-py2.py3-none-any.whl (71kB)</span>
<span style="color: #888888;"> 100% |################################| 71kB 4.2MB/s </span>
<span style="color: #888888;">Collecting retrying (from shub)</span>
<span style="color: #888888;"> Downloading retrying-1.3.3.tar.gz</span>
<span style="color: #888888;">Collecting docker-py (from shub)</span>
<span style="color: #888888;"> Downloading docker_py-1.10.6-py2.py3-none-any.whl (50kB)</span>
<span style="color: #888888;"> 100% |################################| 51kB 4.2MB/s </span>
<span style="color: #888888;">Collecting backports.ssl-match-hostname>=3.5; python_version < "3.5" (from docker-py->shub)</span>
<span style="color: #888888;"> Downloading backports.ssl_match_hostname-3.5.0.1.tar.gz</span>
<span style="color: #888888;">Collecting websocket-client>=0.32.0 (from docker-py->shub)</span>
<span style="color: #888888;"> Downloading websocket_client-0.40.0.tar.gz (196kB)</span>
<span style="color: #888888;"> 100% |################################| 204kB 2.7MB/s </span>
<span style="color: #888888;">Requirement already satisfied: ipaddress>=1.0.16; python_version < "3.3" in /usr/local/lib/python2.7/dist-packages (from docker-py->shub)</span>
<span style="color: #888888;">Collecting docker-pycreds>=0.2.1 (from docker-py->shub)</span>
<span style="color: #888888;"> Downloading docker_pycreds-0.2.1-py2.py3-none-any.whl</span>
<span style="color: #888888;">Building wheels for collected packages: PyYAML, retrying, backports.ssl-match-hostname, websocket-client</span>
<span style="color: #888888;"> Running setup.py bdist_wheel for PyYAML ... done</span>
<span style="color: #888888;"> Stored in directory: /root/.cache/pip/wheels/2c/f7/79/13f3a12cd723892437c0cfbde1230ab4d82947ff7b3839a4fc</span>
<span style="color: #888888;"> Running setup.py bdist_wheel for retrying ... done</span>
<span style="color: #888888;"> Stored in directory: /root/.cache/pip/wheels/d9/08/aa/49f7c109140006ea08a7657640aee3feafb65005bcd5280679</span>
<span style="color: #888888;"> Running setup.py bdist_wheel for backports.ssl-match-hostname ... done</span>
<span style="color: #888888;"> Stored in directory: /root/.cache/pip/wheels/5d/72/36/b2a31507b613967b728edc33378a5ff2ada0f62855b93c5ae1</span>
<span style="color: #888888;"> Running setup.py bdist_wheel for websocket-client ... done</span>
<span style="color: #888888;"> Stored in directory: /root/.cache/pip/wheels/d1/5e/dd/93da015a0ecc8375278b05ad7f0452eff574a044bcea2a95d2</span>
<span style="color: #888888;">Successfully built PyYAML retrying backports.ssl-match-hostname websocket-client</span>
<span style="color: #888888;">Installing collected packages: requests, retrying, scrapinghub, PyYAML, click, backports.ssl-match-hostname, websocket-client, docker-pycreds, docker-py, shub</span>
<span style="color: #888888;">Successfully installed PyYAML-3.12 backports.ssl-match-hostname-3.5.0.1 click-6.6 docker-py-1.10.6 docker-pycreds-0.2.1 requests-2.12.4 retrying-1.3.3 scrapinghub-1.9.0 shub-2.5.0 websocket-client-0.40.0</span>
<span style="color: navy; font-weight: bold;">root@6b82490da131:/home/scrapyd#</span> <span style="color: green;">exit</span>
<span style="color: #888888;">exit</span>
<span style="color: navy; font-weight: bold;">ubuntu@nutthaphongmail:~$</span> docker <span style="color: green;">exec</span> -it scrapyd-server bash
<span style="color: navy; font-weight: bold;">scrapyd@6b82490da131:~$</span>
<span style="color: navy; font-weight: bold;">scrapyd@6b82490da131:~$</span> shub
<span style="color: #888888;">Usage: shub [OPTIONS] COMMAND [ARGS]...</span>
<span style="color: #888888;"> shub is the Scrapinghub command-line client. It allows you to deploy</span>
<span style="color: #888888;"> projects or dependencies, schedule spiders, and retrieve scraped data or</span>
<span style="color: #888888;"> logs without leaving the command line.</span>
<span style="color: #888888;">Options:</span>
<span style="color: #888888;"> --version Show the version and exit.</span>
<span style="color: #888888;"> --help Show this message and exit.</span>
<span style="color: #888888;">Commands:</span>
<span style="color: #888888;"> copy-eggs Sync eggs from one project with other project</span>
<span style="color: #888888;"> deploy Deploy Scrapy project to Scrapy Cloud</span>
<span style="color: #888888;"> deploy-egg [DEPRECATED] Build and deploy egg from source</span>
<span style="color: #888888;"> deploy-reqs [DEPRECATED] Build and deploy eggs from requirements.txt</span>
<span style="color: #888888;"> fetch-eggs Download project eggs from Scrapy Cloud</span>
<span style="color: #888888;"> image Manage project based on custom Docker image</span>
<span style="color: #888888;"> items Fetch items from Scrapy Cloud</span>
<span style="color: #888888;"> log Fetch log from Scrapy Cloud</span>
<span style="color: #888888;"> login Save your Scrapinghub API key</span>
<span style="color: #888888;"> logout Forget saved Scrapinghub API key</span>
<span style="color: #888888;"> migrate-eggs Migrate dash eggs to requirements.txt and project's directory</span>
<span style="color: #888888;"> requests Fetch requests from Scrapy Cloud</span>
<span style="color: #888888;"> schedule Schedule a spider to run on Scrapy Cloud</span>
<span style="color: #888888;"> version Show shub version</span>
<span style="color: #888888;"> For usage and help on a specific command, run it with a --help flag, e.g.:</span>
<span style="color: #888888;"> shub schedule --help</span>
<span style="color: navy; font-weight: bold;">scrapyd@6b82490da131:~$</span>
<span style="color: navy; font-weight: bold;">scrapyd@6b82490da131:~$</span>
<span style="color: navy; font-weight: bold;">scrapyd@6b82490da131:~$</span> <span style="color: green;">pwd</span>
<span style="color: #888888;">/home/scrapyd</span></pre>
<pre style="line-height: 125%; margin: 0;"><pre style="line-height: 16.25px;"><span style="color: navy; font-weight: bold;">scrapyd@6b82490da131:~/projects/PyLearning$</span> <span style="color: green;">cd </span>stack/
<span style="color: navy; font-weight: bold;">scrapyd@6b82490da131:~/projects/PyLearning/stack$</span> cat requirements.txt
<span style="color: #888888;">pymongo==3.4.0</span>
<span style="color: navy; font-weight: bold;">scrapyd@6b82490da131:~/projects/PyLearning/stack$</span>
<span style="color: navy; font-weight: bold;">scrapyd@6b82490da131:~/projects/PyLearning/stack$</span>
<span style="color: navy; font-weight: bold;">scrapyd@6b82490da131:~/projects/PyLearning/stack$</span>
<span style="color: navy; font-weight: bold;">scrapyd@6b82490da131:~/projects/PyLearning/stack$</span> cat scrapinghub.yml
<span style="color: #888888;">projects:</span>
<span style="color: #888888;"> default: 136494</span>
<span style="color: #888888;">requirements_file: requirements.txt</span>
<span style="color: navy; font-weight: bold;">scrapyd@6b82490da131:~/projects/PyLearning/stack$</span> </pre>
<span style="color: navy; font-weight: bold;">scrapyd@6b82490da131:~/projects/PyLearning/stack$</span> shub login
<span style="color: #888888;">-------------------------------------------------------------------------------</span>
<span style="color: #888888;">Welcome to shub version 2!</span>
<span style="color: #888888;">This release contains major updates to how shub is configured, as well as</span>
<span style="color: #888888;">updates to the commands and shub's look & feel.</span>
<span style="color: #888888;">Run 'shub' to get an overview over all available commands, and</span>
<span style="color: #888888;">'shub command --help' to get detailed help on a command. Definitely try the</span>
<span style="color: #888888;">new 'shub items -f [JOBID]' to see items live as they are being scraped!</span>
<span style="color: #888888;">From now on, shub configuration should be done in a file called</span>
<span style="color: #888888;">'scrapinghub.yml', living next to the previously used 'scrapy.cfg' in your</span>
<span style="color: #888888;">Scrapy project directory. Global configuration, for example API keys, should be</span>
<span style="color: #888888;">done in a file called '.scrapinghub.yml' in your home directory.</span>
<span style="color: #888888;">But no worries, shub has automatically migrated your global settings to</span>
<span style="color: #888888;">~/.scrapinghub.yml, and will also automatically migrate your project settings</span>
<span style="color: #888888;">when you run a command within a Scrapy project.</span>
<span style="color: #888888;">Visit http://doc.scrapinghub.com/shub.html for more information on the new</span>
<span style="color: #888888;">configuration format and its benefits.</span>
<span style="color: #888888;">Happy scraping!</span>
<span style="color: #888888;">-------------------------------------------------------------------------------</span>
<span style="color: #888888;">Enter your API key from https://app.scrapinghub.com/account/apikey</span>
<span style="color: #888888;">API key: </span><span style="color: blue;">e4bfa1fd7f8d4d9da817aa112bb82095</span>
<span style="color: #888888;">Validating API key...</span>
<span style="color: #888888;">API key is OK, you are logged in now.</span>
<span style="color: navy; font-weight: bold;">scrapyd@6b82490da131:~/projects/PyLearning/stack$</span>
<span style="color: navy; font-weight: bold;">scrapyd@6b82490da131:~/projects/PyLearning/stack$</span> shub deploy
<span style="color: #888888;">Target project ID: </span><span style="color: blue;">136494</span>
<span style="color: #888888;">Save as default [Y/n]: </span><span style="color: blue;">Y</span>
<span style="color: #888888;">Project 136494 was set as default in scrapinghub.yml. You can deploy to it via 'shub deploy' from now on.</span>
<span style="color: #888888;">Packing version e7d7f6c-inet1</span>
<span style="color: #888888;">Deploying to Scrapy Cloud project "136494"</span>
<span style="color: #888888;">{"spiders": 1, "status": "ok", "project": 136494, "version": "e7d7f6c-inet1"}</span>
<span style="color: #888888;">Run your spiders at: https://app.scrapinghub.com/p/136494/</span>
</pre>
</div>
<br />
* API key <span style="color: blue;">from <span style="color: #888888;">https://app.scrapinghub.com/account/apikey</span></span><br />
** project ID <span style="color: blue;">found on https://app.scrapinghub.com/p/<i>PROJECT_ID</i>/deploy for new project</span><br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />พี่นัทhttp://www.blogger.com/profile/14291406590594822628noreply@blogger.com0tag:blogger.com,1999:blog-2488490746136830746.post-38734391583188760282016-11-22T16:27:00.000+07:002016-11-22T16:33:30.130+07:00Deploy dataset project to scrapyd<br />
<br />
<br />
<br />
<br />
<!-- HTML generated using hilite.me --><br />
<div style="background: #000000; border-width: 0.1em 0.1em 0.1em 0.8em; border: solid gray; overflow: auto; padding: 0.2em 0.6em; width: auto;">
<pre style="line-height: 125%; margin: 0;"><span style="color: navy; font-weight: bold;">scrapyd@a4c2642d74db:~/projects$</span> git clone https://github.com/nutthaphon/PyLearning.git
<span style="color: #888888;">Cloning into 'PyLearning'...</span>
<span style="color: #888888;">remote: Counting objects: 323, done.</span>
<span style="color: #888888;">remote: Compressing objects: 100% (6/6), done.</span>
<span style="color: #888888;">remote: Total 323 (delta 0), reused 0 (delta 0), pack-reused 317</span>
<span style="color: #888888;">Receiving objects: 100% (323/323), 61.27 KiB | 0 bytes/s, done.</span>
<span style="color: #888888;">Resolving deltas: 100% (146/146), done.</span>
<span style="color: #888888;">Checking connectivity... done.</span>
<span style="color: navy; font-weight: bold;">scrapyd@a4c2642d74db:~/projects$</span>
<span style="color: navy; font-weight: bold;">scrapyd@a4c2642d74db:~/projects$</span>
<span style="color: navy; font-weight: bold;">scrapyd@a4c2642d74db:~/projects$</span>
<span style="color: navy; font-weight: bold;">scrapyd@a4c2642d74db:~/projects$</span> <span style="color: #cd00cd;">cd </span>PyLearning/
<span style="color: navy; font-weight: bold;">scrapyd@a4c2642d74db:~/projects/PyLearning$</span> ls
<span style="color: #888888;">Animals CherryPy DJango README.md Scraping dataset decor foo serial test tutorial</span>
<span style="color: navy; font-weight: bold;">scrapyd@a4c2642d74db:~/projects/PyLearning$</span> <span style="color: #cd00cd;">cd </span>dataset/
<span style="color: navy; font-weight: bold;">scrapyd@a4c2642d74db:~/projects/PyLearning/dataset$</span> ls
<span style="color: #888888;">__init__.py __init__.pyc build project.egg-info scrapinghub.yml scrapy.cfg settrade setup.py</span>
<span style="color: navy; font-weight: bold;">scrapyd@a4c2642d74db:~/projects/PyLearning/dataset$</span> sed -i <span style="color: #cd0000;">'s/#url/url/g'</span> scrapy.cfg
<span style="color: navy; font-weight: bold;">scrapyd@a4c2642d74db:~/projects/PyLearning/dataset$</span> cat scrapy.cfg
<span style="color: navy; font-weight: bold;">#</span> Automatically created by: scrapy startproject
<span style="color: navy; font-weight: bold;">#</span>
<span style="color: navy; font-weight: bold;">#</span> For more information about the <span style="color: #3399cc;">[</span>deploy<span style="color: #3399cc;">]</span> section see:
<span style="color: navy; font-weight: bold;">#</span> https://scrapyd.readthedocs.org/en/latest/deploy.html
<span style="color: #888888;">[settings]</span>
<span style="color: #888888;">default = settrade.settings</span>
<span style="color: #888888;">[deploy]</span>
<span style="color: #888888;">url = http://localhost:6800/</span>
<span style="color: #888888;">project = settrade</span>
<span style="color: navy; font-weight: bold;">scrapyd@a4c2642d74db:~/projects/PyLearning/dataset$</span> ls
<span style="color: #888888;">__init__.py __init__.pyc build project.egg-info scrapinghub.yml scrapy.cfg settrade setup.py</span>
<span style="color: navy; font-weight: bold;">scrapyd@a4c2642d74db:~/projects/PyLearning/dataset$</span> <span style="color: #cd00cd;">cd </span>settrade/
<span style="color: navy; font-weight: bold;">scrapyd@a4c2642d74db:~/projects/PyLearning/dataset/settrade$</span> ls
<span style="color: #888888;">DailyStockQuote.py QueryStockSymbol.py ThinkSpeakChannels.py __init__.py items.py scrapinghub.yml settings.pyc</span>
<span style="color: #888888;">DailyStockQuote.pyc QueryStockSymbol.pyc ThinkSpeakChannels.pyc __init__.pyc pipelines.py settings.py spiders</span>
<span style="color: navy; font-weight: bold;">scrapyd@a4c2642d74db:~/projects/PyLearning/dataset/settrade$</span> <span style="color: #cd00cd;">cd </span>spiders/
<span style="color: navy; font-weight: bold;">scrapyd@a4c2642d74db:~/projects/PyLearning/dataset/settrade/spiders$</span> ls
<span style="color: #888888;">SettradeSpider.py SettradeSpider.pyc __init__.py __init__.pyc</span>
<span style="color: navy; font-weight: bold;">scrapyd@a4c2642d74db:~/projects/PyLearning/dataset/settrade/spiders$</span>
<span style="color: navy; font-weight: bold;">scrapyd@a4c2642d74db:~/projects/PyLearning/dataset/settrade/spiders$</span>
<span style="color: navy; font-weight: bold;">scrapyd@a4c2642d74db:~/projects/PyLearning/dataset/settrade/spiders$</span> ls
<span style="color: #888888;">SettradeSpider.py SettradeSpider.pyc __init__.py __init__.pyc</span>
<span style="color: navy; font-weight: bold;">scrapyd@a4c2642d74db:~/projects/PyLearning/dataset/settrade/spiders$</span> <span style="color: #cd00cd;">cd</span> ..
<span style="color: navy; font-weight: bold;">scrapyd@a4c2642d74db:~/projects/PyLearning/dataset/settrade$</span> <span style="color: #cd00cd;">cd</span> ..
<span style="color: navy; font-weight: bold;">scrapyd@a4c2642d74db:~/projects/PyLearning/dataset$</span> ls
<span style="color: #888888;">__init__.py __init__.pyc build project.egg-info scrapinghub.yml scrapy.cfg settrade setup.py</span>
<span style="color: navy; font-weight: bold;">scrapyd@a4c2642d74db:~/projects/PyLearning/dataset$</span>
<span style="color: navy; font-weight: bold;">scrapyd@a4c2642d74db:~/projects/PyLearning/dataset$</span>
<span style="color: navy; font-weight: bold;">scrapyd@a4c2642d74db:~/projects/PyLearning/dataset$</span>
<span style="color: navy; font-weight: bold;">scrapyd@a4c2642d74db:~/projects/PyLearning/dataset$</span>
<span style="color: navy; font-weight: bold;">scrapyd@a4c2642d74db:~/projects/PyLearning/dataset$</span> ~/scrapyd-client-master/scrapyd-client/scrapyd-deploy -l
<span style="color: #888888;">default http://localhost:6800/</span>
<span style="color: navy; font-weight: bold;">scrapyd@a4c2642d74db:~/projects/PyLearning/dataset$</span> ~/scrapyd-client-master/scrapyd-client/scrapyd-deploy default -p dataset
<span style="color: #888888;">Packing version 1479804720</span>
<span style="color: #888888;">Deploying to project "dataset" in http://localhost:6800/addversion.json</span>
<span style="color: #888888;">Server response (200):</span>
<span style="color: #888888;">{"status": "ok", "project": "dataset", "version": "1479804720", "spiders": 1, "node_name": "a4c2642d74db"}</span>
<span style="color: navy; font-weight: bold;">scrapyd@a4c2642d74db:~/projects/PyLearning/dataset$</span>
<span style="color: navy; font-weight: bold;">scrapyd@a4c2642d74db:~/projects/PyLearning/dataset$</span>
<span style="color: navy; font-weight: bold;">scrapyd@a4c2642d74db:~/projects/PyLearning/dataset$</span>
<span style="color: navy; font-weight: bold;">scrapyd@a4c2642d74db:~/projects/PyLearning/dataset$</span>
<span style="color: navy; font-weight: bold;">scrapyd@a4c2642d74db:~/projects/PyLearning/dataset$</span> ~/scrapyd-client-master/scrapyd-client/scrapyd-deploy -L default
<span style="color: #888888;">tutorial</span>
<span style="color: #888888;">dataset</span>
<span style="color: navy; font-weight: bold;">scrapyd@a4c2642d74db:~/projects/PyLearning/dataset$</span> curl http://localhost:6800/listprojects.json
<span style="color: #888888;">{"status": "ok", "projects": ["tutorial", "dataset"], "node_name": "a4c2642d74db"}</span>
<span style="color: navy; font-weight: bold;">scrapyd@a4c2642d74db:~/projects/PyLearning/dataset$</span>
<span style="color: navy; font-weight: bold;">scrapyd@a4c2642d74db:~/projects/PyLearning/dataset$</span>
<span style="color: navy; font-weight: bold;">scrapyd@a4c2642d74db:~/projects/PyLearning/dataset$</span>
<span style="color: navy; font-weight: bold;">scrapyd@a4c2642d74db:~/projects/PyLearning/dataset$</span> curl http://localhost:6800/listspiders.json?project<span style="color: #3399cc;">=</span>dataset
<span style="color: #888888;">{"status": "ok", "spiders": ["settrade_dataset"], "node_name": "a4c2642d74db"}</span>
<span style="color: navy; font-weight: bold;">scrapyd@a4c2642d74db:~/projects/PyLearning/dataset$</span>
<span style="color: navy; font-weight: bold;">scrapyd@a4c2642d74db:~/projects/PyLearning/dataset$</span>
<span style="color: navy; font-weight: bold;">scrapyd@a4c2642d74db:~/projects/PyLearning/dataset$</span> curl http://localhost:6800/schedule.json -d <span style="color: #00cdcd;">project</span><span style="color: #3399cc;">=</span>dataset -d <span style="color: #00cdcd;">spider</span><span style="color: #3399cc;">=</span>settrade_dataset -d <span style="color: #00cdcd;">setting</span><span style="color: #3399cc;">=</span><span style="color: #00cdcd;">DOWNLOAD_DELAY</span><span style="color: #3399cc;">=</span>5 -d <span style="color: #00cdcd;">start_urls</span><span style="color: #3399cc;">=</span>http://www.settrade.com/servlet/IntradayStockChartDataServlet?symbol<span style="color: #3399cc;">=</span>INTUCH,http://www.settrade.com/servlet/IntradayStockChartDataServlet?symbol<span style="color: #3399cc;">=</span>CPF,http://www.settrade.com/servlet/IntradayStockChartDataServlet?symbol<span style="color: #3399cc;">=</span>ADVANC
<span style="color: #888888;">{"status": "ok", "jobid": "c60e4566b09111e68c380242ac110002", "node_name": "a4c2642d74db"}</span>
</pre>
</div>
<br />
<br />
<br />พี่นัทhttp://www.blogger.com/profile/14291406590594822628noreply@blogger.com0tag:blogger.com,1999:blog-2488490746136830746.post-15901049056649495602016-11-22T13:31:00.002+07:002016-11-22T14:28:02.262+07:00Scrapyd deploying your project<br />
<br />
<br />
<br />
<br />
<br />
<!-- HTML generated using hilite.me --><br />
<div style="background: #000000; border-width: 0.1em 0.1em 0.1em 0.8em; border: solid gray; overflow: auto; padding: 0.2em 0.6em; width: auto;">
<pre style="line-height: 125%; margin: 0;"><span style="color: navy; font-weight: bold;">ubuntu@node2:~/Docker/nutthaphon/scrapyd/user$</span> docker run --name scrapyd-server --user scrapyd -it -P nutthaphon/scrapyd:1.1.1 bash
<span style="color: navy; font-weight: bold;">scrapyd@a4c2642d74db:~$</span> <span style="color: #cd00cd;">pwd</span>
<span style="color: #888888;">/home/scrapyd</span>
<span style="color: navy; font-weight: bold;">scrapyd@a4c2642d74db:~$</span> ls
<span style="color: #888888;">master.zip scrapyd-client-master setuptools-28.8.0.zip</span>
<span style="color: navy; font-weight: bold;">scrapyd@a4c2642d74db:~$</span> mkdir projects
<span style="color: navy; font-weight: bold;">scrapyd@a4c2642d74db:~$</span> <span style="color: #cd00cd;">cd </span>projects/
<span style="color: navy; font-weight: bold;">scrapyd@a4c2642d74db:~/projects$</span> scrapy startproject tutorial
<span style="color: #888888;">New Scrapy project 'tutorial', using template directory '/usr/local/lib/python2.7/dist-packages/scrapy/templates/project', created in:</span>
<span style="color: #888888;"> /home/scrapyd/projects/tutorial</span>
<span style="color: #888888;">You can start your first spider with:</span>
<span style="color: #888888;"> cd tutorial</span>
<span style="color: #888888;"> scrapy genspider example example.com</span>
<span style="color: navy; font-weight: bold;">scrapyd@a4c2642d74db:~/projects$</span> <span style="color: #cd00cd;">cd </span>tutorial/
<span style="color: navy; font-weight: bold;">scrapyd@a4c2642d74db:~/projects/tutorial$</span> ls
<span style="color: #888888;">scrapy.cfg tutorial</span>
<span style="color: navy; font-weight: bold;">scrapyd@a4c2642d74db:~/projects/tutorial$</span> vi scrapy.cfg
<span style="color: #888888;">bash: vi: command not found</span>
<span style="color: navy; font-weight: bold;">scrapyd@a4c2642d74db:~/projects/tutorial$</span> cat scrapy.cfg
<span style="color: navy; font-weight: bold;">#</span> Automatically created by: scrapy startproject
<span style="color: navy; font-weight: bold;">#</span>
<span style="color: navy; font-weight: bold;">#</span> For more information about the <span style="color: #3399cc;">[</span>deploy<span style="color: #3399cc;">]</span> section see:
<span style="color: navy; font-weight: bold;">#</span> https://scrapyd.readthedocs.org/en/latest/deploy.html
<span style="color: #888888;">[settings]</span>
<span style="color: #888888;">default = tutorial.settings</span>
<span style="color: #888888;">[deploy]</span>
<span style="color: navy; font-weight: bold;">#</span><span style="color: #00cdcd;">url</span> <span style="color: #3399cc;">=</span> http://localhost:6800/
<span style="color: #888888;">project = tutorial</span>
<span style="color: navy; font-weight: bold;">scrapyd@a4c2642d74db:~/projects/tutorial$</span> sed -i <span style="color: #cd0000;">'s/#utl/url/g'</span> scrapy.cfg
<span style="color: navy; font-weight: bold;">scrapyd@a4c2642d74db:~/projects/tutorial$</span> cat scrapy.cfg
<span style="color: navy; font-weight: bold;">#</span> Automatically created by: scrapy startproject
<span style="color: navy; font-weight: bold;">#</span>
<span style="color: navy; font-weight: bold;">#</span> For more information about the <span style="color: #3399cc;">[</span>deploy<span style="color: #3399cc;">]</span> section see:
<span style="color: navy; font-weight: bold;">#</span> https://scrapyd.readthedocs.org/en/latest/deploy.html
<span style="color: #888888;">[settings]</span>
<span style="color: #888888;">default = tutorial.settings</span>
<span style="color: #888888;">[deploy]</span>
<span style="color: navy; font-weight: bold;">#</span><span style="color: #00cdcd;">url</span> <span style="color: #3399cc;">=</span> http://localhost:6800/
<span style="color: #888888;">project = tutorial</span>
<span style="color: navy; font-weight: bold;">scrapyd@a4c2642d74db:~/projects/tutorial$</span> sed -i <span style="color: #cd0000;">'s/#url/url/g'</span> scrapy.cfg
<span style="color: navy; font-weight: bold;">scrapyd@a4c2642d74db:~/projects/tutorial$</span> cat scrapy.cfg
<span style="color: navy; font-weight: bold;">#</span> Automatically created by: scrapy startproject
<span style="color: navy; font-weight: bold;">#</span>
<span style="color: navy; font-weight: bold;">#</span> For more information about the <span style="color: #3399cc;">[</span>deploy<span style="color: #3399cc;">]</span> section see:
<span style="color: navy; font-weight: bold;">#</span> https://scrapyd.readthedocs.org/en/latest/deploy.html
<span style="color: #888888;">[settings]</span>
<span style="color: #888888;">default = tutorial.settings</span>
<span style="color: #888888;">[deploy]</span>
<span style="color: #888888;">url = http://localhost:6800/</span>
<span style="color: #888888;">project = tutorial</span>
<span style="color: navy; font-weight: bold;">scrapyd@a4c2642d74db:~/projects/tutorial$</span> ~/scrapyd-client-master/scrapyd-client/scrapyd-deploy -l
<span style="color: #888888;">default http://localhost:6800/</span>
<span style="color: navy; font-weight: bold;">scrapyd@a4c2642d74db:~/projects/tutorial$</span> ~/scrapyd-client-master/scrapyd-client/scrapyd-deploy default -p tutorial
<span style="color: #888888;">Packing version 1479799362</span>
<span style="color: #888888;">Deploying to project "tutorial" in http://localhost:6800/addversion.json</span>
<span style="color: #888888;">Deploy failed: <urlopen error [Errno 111] Connection refused></span>
<span style="color: navy; font-weight: bold;">scrapyd@a4c2642d74db:~/projects/tutorial$</span> ~/scrapyd-client-master/scrapyd-client/scrapyd-deploy default -p tutorial
<span style="color: #888888;">Packing version 1479799403</span>
<span style="color: #888888;">Deploying to project "tutorial" in http://localhost:6800/addversion.json</span>
<span style="color: #888888;">Server response (200):</span>
<span style="color: #888888;">{"status": "ok", "project": "tutorial", "version": "1479799403", "spiders": 0, "node_name": "a4c2642d74db"}</span>
<span style="color: navy; font-weight: bold;">scrapyd@a4c2642d74db:~/projects/tutorial$</span> ~/scrapyd-client-master/scrapyd-client/scrapyd-deploy -L default
<span style="color: #888888;">tutorial</span>
</pre>
</div>
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />พี่นัทhttp://www.blogger.com/profile/14291406590594822628noreply@blogger.com0tag:blogger.com,1999:blog-2488490746136830746.post-50947068893625956452016-10-26T17:42:00.000+07:002016-10-27T10:34:24.558+07:00Wildfly and JSF<br />
<br />
<br />
Checking existing JSF subsystem and default module<br />
<div style="background: #000000; border-width: 0.1em 0.1em 0.1em 0.8em; border: solid gray; overflow: auto; padding: 0.2em 0.6em; width: auto;">
<pre style="line-height: 125%; margin: 0;"><span style="color: #888888;">[jboss@69c9222702f3 bin]$ ./jboss-cli.sh </span>
<span style="color: #888888;">You are disconnected at the moment. Type 'connect' to connect to the server or 'help' for the list of supported commands.</span>
<span style="color: #888888;">[disconnected /] connect 10.80.1.209:9990</span>
<span style="color: #888888;">Authenticating against security realm: ManagementRealm</span>
<span style="color: #888888;">Username: admin</span>
<span style="color: #888888;">Password: </span>
<span style="color: #888888;">[standalone@10.80.1.209:9990 /] /subsystem=jsf/:list-active-jsf-impls</span>
<span style="color: #888888;">{</span>
<span style="color: #888888;"> "outcome" => "success",</span>
<span style="color: #888888;"> "result" => ["main"]</span>
<span style="color: #888888;">}</span>
<span style="color: #888888;">[standalone@10.80.1.209:9990 /] /subsystem=jsf/:read-attribute(name=default-jsf-impl-slot)</span>
<span style="color: #888888;">{</span>
<span style="color: #888888;"> "outcome" => "success",</span>
<span style="color: #888888;"> "result" => "main"</span>
<span style="color: #888888;">}</span>
<span style="color: #888888;">[standalone@10.80.1.209:9990 /] </span>
</pre>
</div>
* That show you only one module installed names "main"<br />
<br />
Deploy new one v1.2 and restart Wildfly<br />
<div style="background: #000000; border-width: 0.1em 0.1em 0.1em 0.8em; border: solid gray; overflow: auto; padding: 0.2em 0.6em; width: auto;">
<pre style="line-height: 125%; margin: 0;"><span style="color: #888888;">[standalone@10.80.1.209:9990 /] deploy /install-mojarra-1.2_15.cli</span>
</pre>
</div>
* This cli file, need to build with Maven follow steps in this <a href="https://developer.jboss.org/wiki/StepsToAddAnyNewJSFImplementationOrVersionToWildFly" target="_blank">site</a><br />
-or- Download files from <a href="https://github.com/nutthaphon/wildfly" target="_blank">github</a><br />
<br />
<br />
Verify installed module ,change default to new added and restart Wildfly<br />
<!-- HTML generated using hilite.me --><br />
<div style="background: #000000; border-width: 0.1em 0.1em 0.1em 0.8em; border: solid gray; overflow: auto; padding: 0.2em 0.6em; width: auto;">
<pre style="line-height: 125%; margin: 0;"><span style="color: #888888;">[jboss@69c9222702f3 bin]$ ./jboss-cli.sh </span>
<span style="color: #888888;">You are disconnected at the moment. Type 'connect' to connect to the server or 'help' for the list of supported commands.</span>
<span style="color: #888888;">[disconnected /] connect 10.80.1.209:9990</span>
<span style="color: #888888;">Authenticating against security realm: ManagementRealm</span>
<span style="color: #888888;">Username: admin</span>
<span style="color: #888888;">Password: </span>
<span style="color: #888888;">[standalone@10.80.1.209:9990 /] /subsystem=jsf/:list-active-jsf-impls</span>
<span style="color: #888888;">{</span>
<span style="color: #888888;"> "outcome" => "success",</span>
<span style="color: #888888;"> "result" => [</span>
<span style="color: #888888;"> "mojarra-1.2_15",</span>
<span style="color: #888888;"> "main"</span>
<span style="color: #888888;"> ]</span>
<span style="color: #888888;">}</span>
<span style="color: #888888;">[standalone@10.80.1.209:9990 /] /subsystem=jsf/:read-attribute(name=default-jsf-impl-slot)</span>
<span style="color: #888888;">{</span>
<span style="color: #888888;"> "outcome" => "success",</span>
<span style="color: #888888;"> "result" => "main"</span>
<span style="color: #888888;">}</span>
<span style="color: #888888;">[standalone@10.80.1.209:9990 /] /subsystem=jsf/:write-attribute(name=default-jsf-impl-slot,value=mojarra-1.2_15)</span>
<span style="color: #888888;">{</span>
<span style="color: #888888;"> "outcome" => "success",</span>
<span style="color: #888888;"> "response-headers" => {</span>
<span style="color: #888888;"> "operation-requires-reload" => true,</span>
<span style="color: #888888;"> "process-state" => "reload-required"</span>
<span style="color: #888888;"> }</span>
<span style="color: #888888;">}</span>
<span style="color: #888888;">[standalone@10.80.1.209:9990 /] </span>
</pre>
</div>
<br />
Confirm default to JSF 1.2<br />
<!-- HTML generated using hilite.me --><br />
<div style="background: #000000; border-width: 0.1em 0.1em 0.1em 0.8em; border: solid gray; overflow: auto; padding: 0.2em 0.6em; width: auto;">
<pre style="line-height: 125%; margin: 0;"><span style="color: #888888;">[jboss@69c9222702f3 bin]$ ./jboss-cli.sh </span>
<span style="color: #888888;">You are disconnected at the moment. Type 'connect' to connect to the server or 'help' for the list of supported commands.</span>
<span style="color: #888888;">[disconnected /] connect 10.80.1.209:9990</span>
<span style="color: #888888;">Authenticating against security realm: ManagementRealm</span>
<span style="color: #888888;">Username: admin</span>
<span style="color: #888888;">Password: </span>
<span style="color: #888888;">[standalone@10.80.1.209:9990 /] /subsystem=jsf/:read-attribute(name=default-jsf-impl-slot)</span>
<span style="color: #888888;">{</span>
<span style="color: #888888;"> "outcome" => "success",</span>
<span style="color: #888888;"> "result" => "mojarra-1.2_15"</span>
<span style="color: #888888;">}</span>
</pre>
</div>
<br />
<br />
Test JSF 1.2 Web Application<br />
<br />
<br />
Reference Site : <a href="https://developer.jboss.org/wiki/StepsToAddAnyNewJSFImplementationOrVersionToWildFly" target="_blank">Steps to add any new JSF implementation or version to WildFly</a><br />
<br />
<br />
<br />
<br />
<br />พี่นัทhttp://www.blogger.com/profile/14291406590594822628noreply@blogger.com0tag:blogger.com,1999:blog-2488490746136830746.post-7930029601195678522016-10-11T15:21:00.003+07:002016-10-11T15:21:31.225+07:00JMeter on containers<br />
<br />
<br />
<!-- HTML generated using hilite.me --><br />
<div style="background: #ffffff; border-width: 0.1em 0.1em 0.1em 0.8em; border: solid gray; overflow: auto; padding: 0.2em 0.6em; width: auto;">
<pre style="line-height: 125%; margin: 0;"><span style="color: #c65d09; font-weight: bold;">nutt@nutt-pc:~$</span> docker network create --driver<span style="color: #333333;">=</span>bridge --subnet<span style="color: #333333;">=</span>172.27.0.0/16 <span style="background-color: #fff0f0; color: #666666; font-weight: bold;">\</span>
<span style="color: #c65d09; font-weight: bold;">></span> --ip-range<span style="color: #333333;">=</span>172.27.5.0/24 <span style="background-color: #fff0f0; color: #666666; font-weight: bold;">\</span>
<span style="color: #c65d09; font-weight: bold;">></span> --gateway<span style="color: #333333;">=</span>172.27.5.254 <span style="background-color: #fff0f0; color: #666666; font-weight: bold;">\</span>
<span style="color: #c65d09; font-weight: bold;">></span> my-net
<span style="color: #888888;">1721e834110c184f92ce2ae1008bf5f5dcd735f38a7d35f6bbcc9eebc4d5de8e</span>
<span style="color: #c65d09; font-weight: bold;">nutt@nutt-pc:~$</span> docker network inspect my-net
<span style="color: #888888;">[</span>
<span style="color: #888888;"> {</span>
<span style="color: #888888;"> "Name": "my-net",</span>
<span style="color: #888888;"> "Id": "1721e834110c184f92ce2ae1008bf5f5dcd735f38a7d35f6bbcc9eebc4d5de8e",</span>
<span style="color: #888888;"> "Scope": "local",</span>
<span style="color: #888888;"> "Driver": "bridge",</span>
<span style="color: #888888;"> "EnableIPv6": false,</span>
<span style="color: #888888;"> "IPAM": {</span>
<span style="color: #888888;"> "Driver": "default",</span>
<span style="color: #888888;"> "Options": {},</span>
<span style="color: #888888;"> "Config": [</span>
<span style="color: #888888;"> {</span>
<span style="color: #888888;"> "Subnet": "172.27.0.0/16",</span>
<span style="color: #888888;"> "IPRange": "172.27.5.0/24",</span>
<span style="color: #888888;"> "Gateway": "172.27.5.254"</span>
<span style="color: #888888;"> }</span>
<span style="color: #888888;"> ]</span>
<span style="color: #888888;"> },</span>
<span style="color: #888888;"> "Internal": false,</span>
<span style="color: #888888;"> "Containers": {},</span>
<span style="color: #888888;"> "Options": {},</span>
<span style="color: #888888;"> "Labels": {}</span>
<span style="color: #888888;"> }</span>
<span style="color: #888888;">]</span>
</pre>
</div>
<br />
<!-- HTML generated using hilite.me --><br />
<div style="background: #ffffff; border-width: 0.1em 0.1em 0.1em 0.8em; border: solid gray; overflow: auto; padding: 0.2em 0.6em; width: auto;">
<pre style="line-height: 125%; margin: 0;"><span style="color: #c65d09; font-weight: bold;">nutt@nutt-pc:~$</span> docker run -it --net<span style="color: #333333;">=</span><span style="background-color: #fff0f0;">"my-net"</span> --add-host<span style="color: #333333;">=</span><span style="background-color: #fff0f0;">"jmeter-server:172.27.5.1"</span> --mac-address<span style="color: #333333;">=</span><span style="background-color: #fff0f0;">"9a:17:a0:c7:b4:cb"</span> --ip<span style="color: #333333;">=</span><span style="background-color: #fff0f0;">"172.27.5.1"</span> --name jmeter-server1 ubuntu bash
<span style="color: #c65d09; font-weight: bold;">root@c07696364171:/#</span>
<span style="color: #c65d09; font-weight: bold;">root@c07696364171:/#</span>
<span style="color: #c65d09; font-weight: bold;">root@c07696364171:/#</span>
<span style="color: #c65d09; font-weight: bold;">root@c07696364171:/#</span>
<span style="color: #c65d09; font-weight: bold;">root@c07696364171:/#</span> apt-get update
<span style="color: #888888;">Get:1 http://archive.ubuntu.com/ubuntu xenial InRelease [247 kB]</span>
<span style="color: #888888;">Get:2 http://archive.ubuntu.com/ubuntu xenial-updates InRelease [95.7 kB]</span>
<span style="color: #888888;">Get:3 http://archive.ubuntu.com/ubuntu xenial-security InRelease [94.5 kB]</span>
<span style="color: #888888;">Get:4 http://archive.ubuntu.com/ubuntu xenial/main Sources [1103 kB]</span>
<span style="color: #888888;">Get:5 http://archive.ubuntu.com/ubuntu xenial/restricted Sources [5179 B]</span>
<span style="color: #888888;">Get:6 http://archive.ubuntu.com/ubuntu xenial/universe Sources [9802 kB]</span>
<span style="color: #888888;">Get:7 http://archive.ubuntu.com/ubuntu xenial/main amd64 Packages [1558 kB]</span>
<span style="color: #888888;">Get:8 http://archive.ubuntu.com/ubuntu xenial/restricted amd64 Packages [14.1 kB]</span>
<span style="color: #888888;">Get:9 http://archive.ubuntu.com/ubuntu xenial/universe amd64 Packages [9827 kB]</span>
<span style="color: #888888;">Get:10 http://archive.ubuntu.com/ubuntu xenial-updates/main Sources [244 kB] </span>
<span style="color: #888888;">Get:11 http://archive.ubuntu.com/ubuntu xenial-updates/universe Sources [124 kB]</span>
<span style="color: #888888;">Get:12 http://archive.ubuntu.com/ubuntu xenial-updates/main amd64 Packages [506 kB]</span>
<span style="color: #888888;">Get:13 http://archive.ubuntu.com/ubuntu xenial-updates/universe amd64 Packages [418 kB]</span>
<span style="color: #888888;">Get:14 http://archive.ubuntu.com/ubuntu xenial-security/main Sources [51.4 kB] </span>
<span style="color: #888888;">Get:15 http://archive.ubuntu.com/ubuntu xenial-security/universe Sources [11.1 kB]</span>
<span style="color: #888888;">Get:16 http://archive.ubuntu.com/ubuntu xenial-security/main amd64 Packages [191 kB]</span>
<span style="color: #888888;">Get:17 http://archive.ubuntu.com/ubuntu xenial-security/universe amd64 Packages [52.5 kB]</span>
<span style="color: #888888;">Fetched 24.3 MB in 11s (2109 kB/s) </span>
<span style="color: #888888;">Reading package lists... Done</span>
</pre>
</div>
<br />
<br />
Create <a href="https://github.com/nutthaphon/jmeter/blob/master/Dockerfile" target="_blank">DOCKERFILE</a><br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />พี่นัทhttp://www.blogger.com/profile/14291406590594822628noreply@blogger.com0tag:blogger.com,1999:blog-2488490746136830746.post-74865672103047370152016-09-19T14:28:00.003+07:002016-09-19T14:28:40.284+07:00Python Modules and Packages<br />
<br />
<br />
<br />
Working directory<br />
<!-- HTML generated using hilite.me --><br />
<div style="background: #ffffff; border-width: 0.1em 0.1em 0.1em 0.8em; border: solid gray; overflow: auto; padding: 0.2em 0.6em; width: auto;">
<pre style="line-height: 125%; margin: 0;"><span style="color: #888888;">PyLearning</span>
<span style="color: #888888;">├── Animals</span>
<span style="color: #888888;">├── foo</span>
<span style="color: #888888;">├── README.md</span>
<span style="color: #888888;">└── test</span>
</pre>
</div>
<br />
<!-- HTML generated using hilite.me --><br />
<div style="background: #ffffff; border-width: 0.1em 0.1em 0.1em 0.8em; border: solid gray; overflow: auto; padding: 0.2em 0.6em; width: auto;">
<pre style="line-height: 125%; margin: 0;"><span style="color: #888888;">foo/</span>
<span style="color: #888888;">├── bar.py</span>
<span style="color: #888888;">├── fibo.py</span>
<span style="color: #888888;">├── fibo.pyc</span>
<span style="color: #888888;">├── __init__.py</span>
<span style="color: #888888;">└── __init__.pyc</span>
</pre>
</div>
<br />
<!-- HTML generated using hilite.me --><br />
<div style="background: #ffffff; border-width: 0.1em 0.1em 0.1em 0.8em; border: solid gray; overflow: auto; padding: 0.2em 0.6em; width: auto;">
<pre style="line-height: 125%; margin: 0;"><span style="color: #888888;">test/</span>
<span style="color: #888888;">├── Backwards.py</span>
<span style="color: #888888;">├── Backwards.pyc</span>
<span style="color: #888888;">├── callFibo.py</span>
<span style="color: #888888;">├── callFibo.pyc</span>
<span style="color: #888888;">├── Card.py</span>
<span style="color: #888888;">├── Foo.py</span>
<span style="color: #888888;">├── Foo.pyc</span>
<span style="color: #888888;">├── __init__.py</span>
<span style="color: #888888;">├── __init__.pyc</span>
<span style="color: #888888;">├── mystuff.py</span>
<span style="color: #888888;">├── mystuff.pyc</span>
<span style="color: #888888;">└── support.py</span>
</pre>
</div>
<br />
<br />
<br />
<!-- HTML generated using hilite.me -->fibo.py<br />
<div style="background: #ffffff; border-width: 0.1em 0.1em 0.1em 0.8em; border: solid gray; overflow: auto; padding: 0.2em 0.6em; width: auto;">
<pre style="line-height: 125%; margin: 0;"><span style="color: #008800; font-weight: bold;">def</span> <span style="color: #0066bb; font-weight: bold;">fib</span>(n): <span style="color: #888888;"># write Fibonacci series up to n</span>
a, b <span style="color: #333333;">=</span> <span style="color: #0000dd; font-weight: bold;">0</span>, <span style="color: #0000dd; font-weight: bold;">1</span>
<span style="color: #008800; font-weight: bold;">while</span> b <span style="color: #333333;"><</span> n:
<span style="color: #008800; font-weight: bold;">print</span> b,
a, b <span style="color: #333333;">=</span> b, a<span style="color: #333333;">+</span>b
<span style="color: #008800; font-weight: bold;">def</span> <span style="color: #0066bb; font-weight: bold;">fib2</span>(n): <span style="color: #888888;"># return Fibonacci series up to n</span>
result <span style="color: #333333;">=</span> []
a, b <span style="color: #333333;">=</span> <span style="color: #0000dd; font-weight: bold;">0</span>, <span style="color: #0000dd; font-weight: bold;">1</span>
<span style="color: #008800; font-weight: bold;">while</span> b <span style="color: #333333;"><</span> n:
result<span style="color: #333333;">.</span>append(b)
a, b <span style="color: #333333;">=</span> b, a<span style="color: #333333;">+</span>b
<span style="color: #008800; font-weight: bold;">print</span> result
<span style="color: #008800; font-weight: bold;">def</span> <span style="color: #0066bb; font-weight: bold;">fib3</span>(n): <span style="color: #888888;"># write Fibonacci series up to n</span>
a, b <span style="color: #333333;">=</span> <span style="color: #0000dd; font-weight: bold;">0</span>, <span style="color: #0000dd; font-weight: bold;">1</span>
<span style="color: #008800; font-weight: bold;">while</span> b <span style="color: #333333;"><</span> n:
<span style="color: #008800; font-weight: bold;">print</span> b,
a, b <span style="color: #333333;">=</span> b, a<span style="color: #333333;">+</span>b
</pre>
</div>
<br />
<br />
<br />
<br />
callFibo.py<br />
<!-- HTML generated using hilite.me --><br />
<div style="background: #ffffff; border-width: 0.1em 0.1em 0.1em 0.8em; border: solid gray; overflow: auto; padding: 0.2em 0.6em; width: auto;">
<pre style="line-height: 125%; margin: 0;"><span style="color: #008800; font-weight: bold;">from</span> <span style="color: #0e84b5; font-weight: bold;">foo.fibo</span> <span style="color: #008800; font-weight: bold;">import</span> fib, fib2, fib3
<span style="color: #008800; font-weight: bold;">print</span>(<span style="background-color: #fff0f0;">'Call to fib3()'</span>)
fib3(<span style="color: #0000dd; font-weight: bold;">100</span>)
<span style="color: #008800; font-weight: bold;">print</span>(<span style="background-color: #fff0f0;">'</span><span style="background-color: #fff0f0; color: #666666; font-weight: bold;">\n</span><span style="background-color: #fff0f0;">'</span>)
<span style="color: #008800; font-weight: bold;">print</span>(<span style="background-color: #fff0f0;">'Call to fib2()'</span>)
fib2(<span style="color: #0000dd; font-weight: bold;">100</span>)
<span style="color: #008800; font-weight: bold;">print</span>(<span style="background-color: #fff0f0;">'</span><span style="background-color: #fff0f0; color: #666666; font-weight: bold;">\n</span><span style="background-color: #fff0f0;">'</span>)
<span style="color: #008800; font-weight: bold;">print</span>(<span style="background-color: #fff0f0;">'Call to fib()'</span>)
fib(<span style="color: #0000dd; font-weight: bold;">1000</span>)
<span style="color: #008800; font-weight: bold;">print</span>(<span style="background-color: #fff0f0;">'</span><span style="background-color: #fff0f0; color: #666666; font-weight: bold;">\n</span><span style="background-color: #fff0f0;">'</span>)
<span style="color: #008800; font-weight: bold;">print</span>(<span style="background-color: #fff0f0;">'Call to instance of fib()'</span>)
fib<span style="color: #333333;">=</span>fib
fib(<span style="color: #0000dd; font-weight: bold;">500</span>)
</pre>
</div>
<br />
Output<br />
<!-- HTML generated using hilite.me --><br />
<div style="background: #ffffff; border-width: 0.1em 0.1em 0.1em 0.8em; border: solid gray; overflow: auto; padding: 0.2em 0.6em; width: auto;">
<pre style="line-height: 125%; margin: 0;"><span style="color: #888888;">Call to fib3()</span>
<span style="color: #888888;">1 1 2 3 5 8 13 21 34 55 89 </span>
<span style="color: #888888;">Call to fib2()</span>
<span style="color: #888888;">[1, 1, 2, 3, 5, 8, 13, 21, 34, 55, 89]</span>
<span style="color: #888888;">Call to fib()</span>
<span style="color: #888888;">1 1 2 3 5 8 13 21 34 55 89 144 233 377 610 987 </span>
<span style="color: #888888;">Call to instance of fib()</span>
<span style="color: #888888;">1 1 2 3 5 8 13 21 34 55 89 144 233 377</span>
</pre>
</div>
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />พี่นัทhttp://www.blogger.com/profile/14291406590594822628noreply@blogger.com0tag:blogger.com,1999:blog-2488490746136830746.post-61870352891434769132016-09-14T19:20:00.001+07:002016-09-14T19:20:39.501+07:00Ubuntu Port forwarding<br />
<br />
<br />
Port forwarding<br />
<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjcFtu6_xPYIyR3PbyI7lJkF7m826AkmADSzgzGVdKxSabQaBHugeaiXJMjzv4D4eVZsP2qLQ9OQdLiSdtqJTlKscfzWG13WI8qHkrdPz2WJ5c1bZ2e-mHpTTHgMgf7zcNiHqvdUBpBqDnA/s1600/port_forwarding.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="188" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjcFtu6_xPYIyR3PbyI7lJkF7m826AkmADSzgzGVdKxSabQaBHugeaiXJMjzv4D4eVZsP2qLQ9OQdLiSdtqJTlKscfzWG13WI8qHkrdPz2WJ5c1bZ2e-mHpTTHgMgf7zcNiHqvdUBpBqDnA/s640/port_forwarding.png" width="640" /></a></div>
<br />
<br />
<br />
<br />
<!-- HTML generated using hilite.me --><br />
<div style="background: #ffffff; border-width: 0.1em 0.1em 0.1em 0.8em; border: solid gray; overflow: auto; padding: 0.2em 0.6em; width: auto;">
<pre style="line-height: 125%; margin: 0;">sudo iptables -t nat -I PREROUTING -p tcp -d 192.168.1.106 --dport 9000 -j DNAT --to-destination 10.0.2.129:9000
sudo iptables -t nat -I PREROUTING -p tcp -d 192.168.1.106 --dport 9090 -j DNAT --to-destination 10.0.2.129:9090
sudo iptables -t nat -I PREROUTING -p tcp -d 192.168.1.106 --dport 50070 -j DNAT --to-destination 10.0.2.129:50070
sudo iptables -I FORWARD -m state -d 10.0.2.0/24 --state NEW,RELATED,ESTABLISHED -j ACCEPT
</pre>
</div>
<br />
<br />
<br />
<br />
<br />พี่นัทhttp://www.blogger.com/profile/14291406590594822628noreply@blogger.com0tag:blogger.com,1999:blog-2488490746136830746.post-87989549938557121842016-09-10T23:12:00.001+07:002016-09-11T09:31:27.794+07:00Puppet<br />
<br />
Puppet has 2 distributes :<br />
<br />
1. WEBrick Puppet (Apache) - Naming of services like puppetmaster and puppetagent etc.<br />
2. Puppet Labs (Use in this tutorial)<br />
<br />
<br />
<h3>
Puppet Server: Installing From Packages</h3>
<h4>
Puppet Collections and packages</h4>
$ wget https://apt.puppetlabs.com/puppetlabs-release-pc1-xenial.deb<br />
$ sudo dpkg -i puppetlabs-release-pc1-xenial.deb<br />
$ sudo apt update<br />
$ sudo apt-get install puppetserver<br />
$ sudo systemctl start puppetserver<br />
<h3>
<br />Puppet agent: Linux</h3>
<h4>
Puppet Collections and packages</h4>
$ wget https://apt.puppetlabs.com/puppetlabs-release-pc1-trusty.deb<br />
$ sudo dpkg -i puppetlabs-release-pc1-trusty.deb<br />
<!-- HTML generated using hilite.me --><br />
$ sudo apt-get update<br />
<br />
** Before startup install & configure agent do not forget add Puppet Server(Master) in /etc/hosts at agent side. Default name of Puppet Server is puppet so you map that name to correct IP address.<br />
<br />
<div style="background: #ffffff; border-width: 0.1em 0.1em 0.1em 0.8em; border: solid gray; overflow: auto; padding: 0.2em 0.6em; width: auto;">
<pre style="line-height: 125%; margin: 0;"><span style="color: #c65d09; font-weight: bold;">ubuntu@node1:~$</span> sudo vi /etc/hosts
<span style="color: #c65d09; font-weight: bold;">ubuntu@node1:~$</span>
<span style="color: #c65d09; font-weight: bold;">ubuntu@node1:~$</span> wget https://apt.puppetlabs.com/puppetlabs-release-pc1-trusty.deb
<span style="color: #888888;">--2016-09-10 22:52:47-- https://apt.puppetlabs.com/puppetlabs-release-pc1-trusty.deb</span>
<span style="color: #888888;">Resolving apt.puppetlabs.com (apt.puppetlabs.com)... 198.58.114.168, 2600:3c00::f03c:91ff:fe69:6bf0</span>
<span style="color: #888888;">Connecting to apt.puppetlabs.com (apt.puppetlabs.com)|198.58.114.168|:443... connected.</span>
<span style="color: #888888;">HTTP request sent, awaiting response... 200 OK</span>
<span style="color: #888888;">Length: 13652 (13K) [application/x-debian-package]</span>
<span style="color: #888888;">Saving to: 'puppetlabs-release-pc1-trusty.deb'</span>
<span style="color: #888888;">100%[================================================================================================================>] 13,652 44.3KB/s in 0.3s </span>
<span style="color: #888888;">2016-09-10 22:52:49 (44.3 KB/s) - 'puppetlabs-release-pc1-trusty.deb' saved [13652/13652]</span>
<span style="color: #c65d09; font-weight: bold;">ubuntu@node1:~$</span> sudo dpkg -i puppetlabs-release-pc1-trusty.deb
<span style="color: #888888;">Selecting previously unselected package puppetlabs-release-pc1.</span>
<span style="color: #888888;">(Reading database ... 57362 files and directories currently installed.)</span>
<span style="color: #888888;">Preparing to unpack puppetlabs-release-pc1-trusty.deb ...</span>
<span style="color: #888888;">Unpacking puppetlabs-release-pc1 (1.1.0-2trusty) ...</span>
<span style="color: #888888;">Setting up puppetlabs-release-pc1 (1.1.0-2trusty) ...</span>
<span style="color: #c65d09; font-weight: bold;">ubuntu@node1:~$</span> sudo apt-get update
<span style="color: #888888;">Hit https://apt.dockerproject.org ubuntu-trusty InRelease</span>
<span style="color: #888888;">Ign http://apt.puppetlabs.com trusty InRelease </span>
<span style="color: #888888;">Hit https://apt.dockerproject.org ubuntu-trusty/main amd64 Packages </span>
<span style="color: #888888;">Ign http://archive.ubuntu.com trusty InRelease </span>
<span style="color: #888888;">Get:1 https://apt.dockerproject.org ubuntu-trusty/main Translation-en</span>
<span style="color: #888888;">Get:2 http://apt.puppetlabs.com trusty Release.gpg [841 B] </span>
<span style="color: #888888;">Ign https://apt.dockerproject.org ubuntu-trusty/main Translation-en </span>
<span style="color: #888888;">Get:3 http://archive.ubuntu.com trusty-updates InRelease [65.9 kB]</span>
<span style="color: #888888;">Get:4 http://apt.puppetlabs.com trusty Release [54.2 kB] </span>
<span style="color: #888888;">Get:5 http://archive.ubuntu.com trusty-security InRelease [65.9 kB]</span>
<span style="color: #888888;">Get:6 http://apt.puppetlabs.com trusty/PC1 amd64 Packages [15.6 kB]</span>
<span style="color: #888888;">Hit http://archive.ubuntu.com trusty Release.gpg </span>
<span style="color: #888888;">Get:7 http://archive.ubuntu.com trusty-updates/main amd64 Packages [889 kB]</span>
<span style="color: #888888;">Get:8 http://archive.ubuntu.com trusty-updates/restricted amd64 Packages [15.9 kB]</span>
<span style="color: #888888;">Get:9 http://archive.ubuntu.com trusty-updates/universe amd64 Packages [373 kB]</span>
<span style="color: #888888;">Get:10 http://archive.ubuntu.com trusty-updates/multiverse amd64 Packages [14.8 kB]</span>
<span style="color: #888888;">Ign http://apt.puppetlabs.com trusty/PC1 Translation-en </span>
<span style="color: #888888;">Get:11 http://archive.ubuntu.com trusty-updates/main Translation-en [431 kB]</span>
<span style="color: #888888;">Get:12 http://archive.ubuntu.com trusty-updates/multiverse Translation-en [7661 B]</span>
<span style="color: #888888;">Get:13 http://archive.ubuntu.com trusty-updates/restricted Translation-en [3699 B]</span>
<span style="color: #888888;">Get:14 http://archive.ubuntu.com trusty-updates/universe Translation-en [197 kB]</span>
<span style="color: #888888;">Get:15 http://archive.ubuntu.com trusty-security/main amd64 Packages [524 kB] </span>
<span style="color: #888888;">Get:16 http://archive.ubuntu.com trusty-security/restricted amd64 Packages [13.0 kB]</span>
<span style="color: #888888;">Get:17 http://archive.ubuntu.com trusty-security/universe amd64 Packages [136 kB]</span>
<span style="color: #888888;">Get:18 http://archive.ubuntu.com trusty-security/multiverse amd64 Packages [4990 B]</span>
<span style="color: #888888;">Get:19 http://archive.ubuntu.com trusty-security/main Translation-en [288 kB] </span>
<span style="color: #888888;">Get:20 http://archive.ubuntu.com trusty-security/multiverse Translation-en [2570 B]</span>
<span style="color: #888888;">Get:21 http://archive.ubuntu.com trusty-security/restricted Translation-en [3206 B]</span>
<span style="color: #888888;">Get:22 http://archive.ubuntu.com trusty-security/universe Translation-en [81.3 kB]</span>
<span style="color: #888888;">Hit http://archive.ubuntu.com trusty Release </span>
<span style="color: #888888;">Hit http://archive.ubuntu.com trusty/main amd64 Packages </span>
<span style="color: #888888;">Hit http://archive.ubuntu.com trusty/restricted amd64 Packages </span>
<span style="color: #888888;">Hit http://archive.ubuntu.com trusty/universe amd64 Packages </span>
<span style="color: #888888;">Hit http://archive.ubuntu.com trusty/multiverse amd64 Packages </span>
<span style="color: #888888;">Hit http://archive.ubuntu.com trusty/main Translation-en </span>
<span style="color: #888888;">Hit http://archive.ubuntu.com trusty/multiverse Translation-en </span>
<span style="color: #888888;">Hit http://archive.ubuntu.com trusty/restricted Translation-en </span>
<span style="color: #888888;">Hit http://archive.ubuntu.com trusty/universe Translation-en </span>
<span style="color: #888888;">Fetched 3187 kB in 18s (171 kB/s) </span>
<span style="color: #888888;">Reading package lists... Done</span>
<span style="color: #c65d09; font-weight: bold;">ubuntu@node1:~$</span> sudo apt-get install puppet-agent
<span style="color: #888888;">Reading package lists... Done</span>
<span style="color: #888888;">Building dependency tree </span>
<span style="color: #888888;">Reading state information... Done</span>
<span style="color: #888888;">The following NEW packages will be installed:</span>
<span style="color: #888888;"> puppet-agent</span>
<span style="color: #888888;">0 upgraded, 1 newly installed, 0 to remove and 12 not upgraded.</span>
<span style="color: #888888;">Need to get 15.1 MB of archives.</span>
<span style="color: #888888;">After this operation, 81.8 MB of additional disk space will be used.</span>
<span style="color: #888888;">Get:1 http://apt.puppetlabs.com/ trusty/PC1 puppet-agent amd64 1.6.2-1trusty [15.1 MB]</span>
<span style="color: #888888;">Fetched 15.1 MB in 0s (60.7 MB/s) </span>
<span style="color: #888888;">perl: warning: Setting locale failed.</span>
<span style="color: #888888;">perl: warning: Please check that your locale settings:</span>
<span style="color: #888888;"> LANGUAGE = (unset),</span>
<span style="color: #888888;"> LC_ALL = (unset),</span>
<span style="color: #888888;"> LC_TIME = "th_TH.UTF-8",</span>
<span style="color: #888888;"> LC_MONETARY = "th_TH.UTF-8",</span>
<span style="color: #888888;"> LC_ADDRESS = "th_TH.UTF-8",</span>
<span style="color: #888888;"> LC_TELEPHONE = "th_TH.UTF-8",</span>
<span style="color: #888888;"> LC_NAME = "th_TH.UTF-8",</span>
<span style="color: #888888;"> LC_MEASUREMENT = "th_TH.UTF-8",</span>
<span style="color: #888888;"> LC_IDENTIFICATION = "th_TH.UTF-8",</span>
<span style="color: #888888;"> LC_NUMERIC = "th_TH.UTF-8",</span>
<span style="color: #888888;"> LC_PAPER = "th_TH.UTF-8",</span>
<span style="color: #888888;"> LANG = "en_US.UTF-8"</span>
<span style="color: #888888;"> are supported and installed on your system.</span>
<span style="color: #888888;">perl: warning: Falling back to the standard locale ("C").</span>
<span style="color: #888888;">locale: Cannot set LC_ALL to default locale: No such file or directory</span>
<span style="color: #888888;">Selecting previously unselected package puppet-agent.</span>
<span style="color: #888888;">(Reading database ... 57367 files and directories currently installed.)</span>
<span style="color: #888888;">Preparing to unpack .../puppet-agent_1.6.2-1trusty_amd64.deb ...</span>
<span style="color: #888888;">Unpacking puppet-agent (1.6.2-1trusty) ...</span>
<span style="color: #888888;">Processing triggers for ureadahead (0.100.0-16) ...</span>
<span style="color: #888888;">Setting up puppet-agent (1.6.2-1trusty) ...</span>
<span style="color: #888888;">update-rc.d: warning: start runlevel arguments (none) do not match pxp-agent Default-Start values (2 3 4 5)</span>
<span style="color: #888888;">update-rc.d: warning: stop runlevel arguments (none) do not match pxp-agent Default-Stop values (0 1 6)</span>
<span style="color: #888888;">Processing triggers for ureadahead (0.100.0-16) ...</span>
<span style="color: #c65d09; font-weight: bold;">ubuntu@node1:~$</span> sudo /opt/puppetlabs/bin/puppet resource service puppet <span style="color: #996633;">ensure</span><span style="color: #333333;">=</span>running <span style="color: #007020;">enable</span><span style="color: #333333;">=</span><span style="color: #007020;">true</span>
<span style="color: #888888;">2016-09-10 22:54:37.681893 WARN puppetlabs.facter - locale environment variables were bad; continuing with LANG=C LC_ALL=C</span>
<span style="color: #888888;">Notice: /Service[puppet]/ensure: ensure changed 'stopped' to 'running'</span>
<span style="color: #888888;">service { 'puppet':</span>
<span style="color: #888888;"> ensure => 'running',</span>
<span style="color: #888888;"> enable => 'true',</span>
<span style="color: #888888;">}</span>
<span style="color: #c65d09; font-weight: bold;">ubuntu@node1:~$</span> sudo /opt/puppetlabs/bin/puppet agent --test
<span style="color: #888888;">2016-09-10 22:57:11.396529 WARN puppetlabs.facter - locale environment variables were bad; continuing with LANG=C LC_ALL=C</span>
<span style="color: #888888;">Info: Using configured environment 'production'</span>
<span style="color: #888888;">Info: Retrieving pluginfacts</span>
<span style="color: #888888;">Info: Retrieving plugin</span>
<span style="color: #888888;">Info: Caching catalog for node1</span>
<span style="color: #888888;">Info: Applying configuration version '1473523040'</span>
<span style="color: #888888;">Notice: Applied catalog in 0.24 seconds</span>
</pre>
</div>
<br />
Sign certificates on the CA master<br />
<br />
<!-- HTML generated using hilite.me --><br />
<div style="background: #ffffff; border-width: 0.1em 0.1em 0.1em 0.8em; border: solid gray; overflow: auto; padding: 0.2em 0.6em; width: auto;">
<pre style="line-height: 125%; margin: 0;"><span style="color: #c65d09; font-weight: bold;">nutt@nutt-pc:~/Downloads$</span> sudo /opt/puppetlabs/bin/puppet cert list
<span style="color: #888888;">Warning: Facter: Could not process routing table entry: Expected a destination followed by key/value pairs, got '192.168.122.0/24 dev virbr0 proto kernel scope link src 192.168.122.1 linkdown'</span>
<span style="color: #888888;"> "node1" (SHA256) CE:54:74:42:95:4A:C5:44:20:90:23:26:3C:63:F0:4D:71:79:12:BC:06:CC:0A:6A:ED:DE:E4:BD:AA:77:C2:A3</span>
<span style="color: #c65d09; font-weight: bold;">nutt@nutt-pc:~/Downloads$</span> sudo /opt/puppetlabs/bin/puppet cert sign node1
<span style="color: #888888;">Warning: Facter: Could not process routing table entry: Expected a destination followed by key/value pairs, got '192.168.122.0/24 dev virbr0 proto kernel scope link src 192.168.122.1 linkdown'</span>
<span style="color: #888888;">Signing Certificate Request for:</span>
<span style="color: #888888;"> "node1" (SHA256) CE:54:74:42:95:4A:C5:44:20:90:23:26:3C:63:F0:4D:71:79:12:BC:06:CC:0A:6A:ED:DE:E4:BD:AA:77:C2:A3</span>
<span style="color: #888888;">Notice: Signed certificate request for node1</span>
<span style="color: #888888;">Notice: Removing file Puppet::SSL::CertificateRequest node1 at '/etc/puppetlabs/puppet/ssl/ca/requests/node1.pem'</span>
</pre>
</div>
<br />
<br />
<br />
<h3>
Master-Agent Simple Setup</h3>
<br />
Main file /etc/puppetlabs/code/environments/production/manifests/site.pp<br />
<br />
All node effects<br />
<!-- HTML generated using hilite.me --><br />
<div style="background: #ffffff; border-width: 0.1em 0.1em 0.1em 0.8em; border: solid gray; overflow: auto; padding: 0.2em 0.6em; width: auto;">
<pre style="line-height: 125%; margin: 0;"><span style="color: #008800; font-weight: bold;">file</span> {<span style="background-color: #fff0f0;">'/tmp/example-ip'</span>:<span style="color: #888888;"> # resource type file and filename</span>
<span style="color: #0000cc;">ensure</span> <span style="color: #333333;">=></span> <span style="color: #008800; font-weight: bold;">present</span>,<span style="color: #888888;"> # make sure it exists</span>
<span style="color: #0000cc;">mode</span> <span style="color: #333333;">=></span> <span style="background-color: #fff0f0;">'0644'</span>,<span style="color: #888888;"> # file permissions</span>
<span style="color: #0000cc;">content</span> <span style="color: #333333;">=></span> <span style="background-color: #fff0f0;">"Here is my Public IP Address: ${ipaddress_eth0}.\n"</span>,<span style="color: #888888;"> # note the ipaddress_eth0 fact</span>
}
</pre>
</div>
<br />
Specific node effects<br />
<!-- HTML generated using hilite.me --><br />
<div style="background: #ffffff; border-width: 0.1em 0.1em 0.1em 0.8em; border: solid gray; overflow: auto; padding: 0.2em 0.6em; width: auto;">
<pre style="line-height: 125%; margin: 0;"><span style="color: #008800; font-weight: bold;">node</span> <span style="background-color: #fff0f0;">'ns1'</span>, <span style="background-color: #fff0f0;">'ns2'</span> {<span style="color: #888888;"> # applies to ns1 and ns2 nodes</span>
<span style="color: #008800; font-weight: bold;">file</span> {<span style="background-color: #fff0f0;">'/tmp/dns'</span>:<span style="color: #888888;"> # resource type file and filename</span>
<span style="color: #0000cc;">ensure</span> <span style="color: #333333;">=></span> <span style="color: #008800; font-weight: bold;">present</span>,<span style="color: #888888;"> # make sure it exists</span>
<span style="color: #0000cc;">mode</span> <span style="color: #333333;">=></span> <span style="background-color: #fff0f0;">'0644'</span>,
<span style="color: #0000cc;">content</span> <span style="color: #333333;">=></span> <span style="background-color: #fff0f0;">"Only DNS servers get this file.\n"</span>,
}
}
<span style="color: #008800; font-weight: bold;">node</span> <span style="color: #008800; font-weight: bold;">default</span> {}<span style="color: #888888;"> # applies to nodes that aren't explicitly defined</span>
</pre>
</div>
<br />
<br />
Resource propagate to nodes depend on their schedule or take effect them immediately with 'puppet agent --test'<br />
As a result of above configuration, all node will have a file name 'dns' and 'example-ip' in /tmp directory<br />
<br />
<h3>
Puppet Apply (standalone run puppet file)</h3>
<br />
<h4>
Optional, set environment variable: only root user can run Puppet</h4>
PATH=/opt/puppetlabs/bin:$PATH;export PATH<br />
<br />
docker_example.pp<br />
<br />
<!-- HTML generated using hilite.me --><br />
<div style="background: #ffffff; border-width: 0.1em 0.1em 0.1em 0.8em; border: solid gray; overflow: auto; padding: 0.2em 0.6em; width: auto;">
<pre style="line-height: 125%; margin: 0;"><span style="color: #008800; font-weight: bold;">include</span> <span style="background-color: #fff0f0;">'docker'</span>
<span style="color: #0000cc;">docker</span>::<span style="color: #0000cc;">run</span> { <span style="background-color: #fff0f0;">'helloworld'</span>:
<span style="color: #0000cc;">image</span> <span style="color: #333333;">=></span> <span style="background-color: #fff0f0;">'ubuntu:precise'</span>,
<span style="color: #0000cc;">command</span> <span style="color: #333333;">=></span> <span style="background-color: #fff0f0;">'/bin/sh -c "while true; do echo hello world; sleep 1; done"'</span>,
}
</pre>
</div>
<div>
<br /></div>
<br />
<br />
<br />
<h3>
Troubleshooting</h3>
<br />
Some of client may have an error when run Puppet command like below:<br />
<br />
<!-- HTML generated using hilite.me --><br />
<div style="background: #ffffff; border-width: 0.1em 0.1em 0.1em 0.8em; border: solid gray; overflow: auto; padding: 0.2em 0.6em; width: auto;">
<pre style="line-height: 125%; margin: 0;"><span style="color: #c65d09; font-weight: bold;">root@node2:~/Docker/puppet#</span> facter
<span style="color: #888888;">2016-09-11 00:21:25.117804 WARN puppetlabs.facter - locale environment variables were bad; continuing with LANG=C LC_ALL=C</span>
<span style="color: #888888;">2016-09-11 00:21:25.146927 FATAL puppetlabs.facter - unhandled exception: boost::filesystem::current_path: No such file or directory</span>
<span style="color: #c65d09; font-weight: bold;">root@node2:~/Docker/puppet#</span> puppet agent --test
<span style="color: #888888;">2016-09-11 00:23:03.006097 WARN puppetlabs.facter - locale environment variables were bad; continuing with LANG=C LC_ALL=C</span>
<span style="color: #888888;">terminate called after throwing an instance of 'boost::filesystem::filesystem_error'</span>
<span style="color: #888888;"> what(): boost::filesystem::current_path: No such file or directory</span>
<span style="color: #888888;">Aborted (core dumped)</span>
</pre>
</div>
<br />
This because server do not have some library such as '<span style="color: blue;">libboost-filesystem-dev</span>'<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />พี่นัทhttp://www.blogger.com/profile/14291406590594822628noreply@blogger.com1tag:blogger.com,1999:blog-2488490746136830746.post-66054321690768439002016-09-05T15:45:00.003+07:002016-09-05T15:45:16.902+07:00Apache Nutch<br />
<br />
<!-- HTML generated using hilite.me --><br />
<div style="background: #ffffff; border-width: 0.1em 0.1em 0.1em 0.8em; border: solid gray; overflow: auto; padding: 0.2em 0.6em; width: auto;">
<pre style="line-height: 125%; margin: 0;"><span style="color: #c65d09; font-weight: bold;">ubuntu@node2:~$</span> docker <span style="color: #007020;">exec</span> -it hbase bash
<span style="color: #c65d09; font-weight: bold;">root@45883500b170:/#</span>
<span style="color: #c65d09; font-weight: bold;">root@45883500b170:/#</span>
<span style="color: #c65d09; font-weight: bold;">root@45883500b170:/#</span>
<span style="color: #c65d09; font-weight: bold;">root@45883500b170:/#</span> useradd nutch -m -s /bin/bash
<span style="color: #c65d09; font-weight: bold;">root@45883500b170:/#</span> passwd nutch
<span style="color: #888888;">Enter new UNIX password: </span>
<span style="color: #888888;">Retype new UNIX password: </span>
<span style="color: #888888;">passwd: password updated successfully</span>
<span style="color: #c65d09; font-weight: bold;">root@45883500b170:/#</span>
<span style="color: #c65d09; font-weight: bold;">root@45883500b170:/#</span>
<span style="color: #c65d09; font-weight: bold;">root@45883500b170:/#</span>
<span style="color: #c65d09; font-weight: bold;">root@45883500b170:/#</span> <span style="color: #007020;">exit</span>
<span style="color: #888888;">exit</span>
<span style="color: #c65d09; font-weight: bold;">ubuntu@node2:~$</span> docker <span style="color: #007020;">exec</span> -it --user nutch hbase bash
<span style="color: #c65d09; font-weight: bold;">nutch@45883500b170:/$</span>
<span style="color: #c65d09; font-weight: bold;">nutch@45883500b170:/$</span>
<span style="color: #c65d09; font-weight: bold;">nutch@45883500b170:/$</span> <span style="color: #007020;">pwd</span>
<span style="color: #888888;">/</span>
<span style="color: #c65d09; font-weight: bold;">nutch@45883500b170:/$</span> <span style="color: #007020;">cd</span>
<span style="color: #c65d09; font-weight: bold;">nutch@45883500b170:~$</span><span style="color: #007020;"> pwd</span>
<span style="color: #888888;">/home/nutch</span>
<span style="color: #c65d09; font-weight: bold;">nutch@45883500b170:~$</span> tar xzvf /software/apache-nutch-2.3.1-src.tar.gz
<span style="color: #888888;">apache-nutch-2.3.1/conf/</span>
<span style="color: #888888;">apache-nutch-2.3.1/docs/</span>
<span style="color: #888888;">apache-nutch-2.3.1/docs/api/</span>
<span style="color: #888888;">apache-nutch-2.3.1/docs/api/org/</span>
<span style="color: #888888;">apache-nutch-2.3.1/docs/api/org/apache/</span>
<span style="color: #888888;">apache-nutch-2.3.1/docs/api/org/apache/nutch/</span>
<span style="color: #888888;">apache-nutch-2.3.1/docs/api/org/apache/nutch/analysis/</span>
<span style="color: #888888;">apache-nutch-2.3.1/docs/api/org/apache/nutch/analysis/lang/</span>
<span style="color: #888888;">apache-nutch-2.3.1/docs/api/org/apache/nutch/analysis/lang/class-use/</span>
<span style="color: #888888;">apache-nutch-2.3.1/docs/api/org/apache/nutch/api/</span>
<span style="color: #888888;">apache-nutch-2.3.1/docs/api/org/apache/nutch/api/class-use/</span>
<span style="color: #888888;">apache-nutch-2.3.1/docs/api/org/apache/nutch/api/impl/</span>
<span style="color: #888888;">apache-nutch-2.3.1/docs/api/org/apache/nutch/api/impl/class-use/</span>
<span style="color: #888888;">apache-nutch-2.3.1/docs/api/org/apache/nutch/api/impl/db/</span>
<span style="color: #888888;">apache-nutch-2.3.1/docs/api/org/apache/nutch/api/impl/db/class-use/</span>
<span style="color: #888888;">apache-nutch-2.3.1/docs/api/org/apache/nutch/api/misc/</span>
<span style="color: #888888;">apache-nutch-2.3.1/docs/api/org/apache/nutch/api/misc/class-use/</span>
<span style="color: #888888;">apache-nutch-2.3.1/docs/api/org/apache/nutch/api/model/</span>
<span style="color: #888888;">apache-nutch-2.3.1/docs/api/org/apache/nutch/api/model/request/</span>
<span style="color: #888888;">apache-nutch-2.3.1/docs/api/org/apache/nutch/api/model/request/class-use/</span>
<span style="color: #888888;">apache-nutch-2.3.1/docs/api/org/apache/nutch/api/model/response/</span>
<span style="color: #888888;">apache-nutch-2.3.1/docs/api/org/apache/nutch/api/model/response/class-use/</span>
<span style="color: #888888;">apache-nutch-2.3.1/docs/api/org/apache/nutch/api/resources/</span>
</pre>
<pre style="line-height: 125%; margin: 0;"><span style="color: #888888;">...</span></pre>
</div>
<br />
$NUTCH_HOME/ivy/ivy.xml :<br />
<!-- HTML generated using hilite.me --><br />
<div style="background: #ffffff; border-width: 0.1em 0.1em 0.1em 0.8em; border: solid gray; overflow: auto; padding: 0.2em 0.6em; width: auto;">
<pre style="line-height: 125%; margin: 0;"><span style="color: #007700;"><dependency</span> <span style="color: #0000cc;">org=</span><span style="background-color: #fff0f0;">"org.apache.gora"</span> <span style="color: #0000cc;">name=</span><span style="background-color: #fff0f0;">"gora-hbase"</span> <span style="color: #0000cc;">rev=</span><span style="background-color: #fff0f0;">"0.6.1"</span> <span style="color: #0000cc;">conf=</span><span style="background-color: #fff0f0;">"*->default"</span> <span style="color: #007700;">/></span>
<span style="color: #007700;"><dependency</span> <span style="color: #0000cc;">org=</span><span style="background-color: #fff0f0;">"org.apache.hbase"</span> <span style="color: #0000cc;">name=</span><span style="background-color: #fff0f0;">"hbase-common"</span> <span style="color: #0000cc;">rev=</span><span style="background-color: #fff0f0;">"0.98.8-hadoop2"</span> <span style="color: #0000cc;">conf=</span><span style="background-color: #fff0f0;">"*->default"</span> <span style="color: #007700;">/></span>
</pre>
</div>
<br />
<br />
$NUTCH_HOME/conf/gora.properties :<br />
<!-- HTML generated using hilite.me --><br />
<div style="background: #ffffff; border-width: 0.1em 0.1em 0.1em 0.8em; border: solid gray; overflow: auto; padding: 0.2em 0.6em; width: auto;">
<pre style="line-height: 125%; margin: 0;">############################
# HBaseStore properties #
############################
gora.datastore.default=org.apache.gora.hbase.store.HBaseStore
gora.datastore.autocreateschema=true
gora.datastore.scanner.caching=1000
hbase.client.autoflush.default=false
</pre>
</div>
<br />
<!-- HTML generated using hilite.me --><br />
<div style="background: #ffffff; border-width: 0.1em 0.1em 0.1em 0.8em; border: solid gray; overflow: auto; padding: 0.2em 0.6em; width: auto;">
<pre style="line-height: 125%; margin: 0;"><span style="color: #c65d09; font-weight: bold;">nutch@45883500b170:~/apache-nutch-2.3.1$</span> ant clean
<span style="color: #888888;">Buildfile: /home/nutch/apache-nutch-2.3.1/build.xml</span>
<span style="color: #888888;">Trying to override old definition of task javac</span>
<span style="color: #888888;"> [taskdef] Could not load definitions from resource org/sonar/ant/antlib.xml. It could not be found.</span>
<span style="color: #888888;">clean-build:</span>
<span style="color: #888888;"> [delete] Deleting directory /home/nutch/apache-nutch-2.3.1/build</span>
<span style="color: #888888;">clean-lib:</span>
<span style="color: #888888;">clean-dist:</span>
<span style="color: #888888;">clean-runtime:</span>
<span style="color: #888888;">clean:</span>
<span style="color: #888888;">BUILD SUCCESSFUL</span>
<span style="color: #888888;">Total time: 0 seconds</span>
<span style="color: #c65d09; font-weight: bold;">nutch@45883500b170:~/apache-nutch-2.3.1$</span> ant runtime
<span style="color: #888888;">Buildfile: /home/nutch/apache-nutch-2.3.1/build.xml</span>
<span style="color: #888888;">Trying to override old definition of task javac</span>
<span style="color: #888888;"> [taskdef] Could not load definitions from resource org/sonar/ant/antlib.xml. It could not be found.</span>
<span style="color: #888888;">ivy-probe-antlib:</span>
<span style="color: #888888;">ivy-download:</span>
<span style="color: #888888;"> [taskdef] Could not load definitions from resource org/sonar/ant/antlib.xml. It could not be found.</span>
<span style="color: #888888;">ivy-download-unchecked:</span>
<span style="color: #888888;">ivy-init-antlib:</span>
<span style="color: #888888;">ivy-init:</span>
<span style="color: #888888;">init:</span>
<span style="color: #888888;"> [mkdir] Created dir: /home/nutch/apache-nutch-2.3.1/build</span>
<span style="color: #888888;"> [mkdir] Created dir: /home/nutch/apache-nutch-2.3.1/build/classes</span>
<span style="color: #888888;"> [mkdir] Created dir: /home/nutch/apache-nutch-2.3.1/build/release</span>
<span style="color: #888888;"> [mkdir] Created dir: /home/nutch/apache-nutch-2.3.1/build/test</span>
<span style="color: #888888;"> [mkdir] Created dir: /home/nutch/apache-nutch-2.3.1/build/test/classes</span>
<span style="color: #888888;">clean-lib:</span>
<span style="color: #888888;">resolve-default:</span>
<span style="color: #888888;">[ivy:resolve] :: Apache Ivy 2.4.0 - 20141213170938 :: http://ant.apache.org/ivy/ ::</span>
<span style="color: #888888;">[ivy:resolve] :: loading settings :: file = /home/nutch/apache-nutch-2.3.1/ivy/ivysettings.xml</span>
<span style="color: #888888;">[ivy:resolve] downloading http://repo1.maven.org/maven2/org/apache/solr/solr-solrj/4.6.0/solr-solrj-4.6.0.jar ...</span>
<span style="color: #888888;">[ivy:resolve] ...........</span>
<span style="color: #888888;">[ivy:resolve] .............................</span>
<span style="color: #888888;">[ivy:resolve] . (393kB)</span>
<span style="color: #888888;">[ivy:resolve] .. (0kB)</span>
<span style="color: #c65d09; font-weight: bold;">[ivy:resolve] [SUCCESSFUL ] org.apache.solr#</span>solr-solrj;4.6.0!solr-solrj.jar <span style="color: #333333;">(</span>4382ms<span style="color: #333333;">)</span>
<span style="color: #888888;">[ivy:resolve] downloading http://repo1.maven.org/maven2/org/apache/hadoop/hadoop-common/2.5.2/hadoop-common-2.5.2.jar ...</span>
<span style="color: #888888;">[ivy:resolve] .................</span>
<span style="color: #888888;">[ivy:resolve] ...............................</span>
<span style="color: #888888;">[ivy:resolve] ................</span>
<span style="color: #888888;">[ivy:resolve] .......................</span>
<span style="color: #888888;">[ivy:resolve] ........................</span>
<span style="color: #888888;">[ivy:resolve] ........................</span>
<span style="color: #888888;">[ivy:resolve] .........................</span>
<span style="color: #888888;">[ivy:resolve] .......................</span>
<span style="color: #888888;">[ivy:resolve] ............................</span>
<span style="color: #888888;">[ivy:resolve] ......................</span>
<span style="color: #888888;">[ivy:resolve] ............................</span>
<span style="color: #888888;">[ivy:resolve] ......................</span>
<span style="color: #888888;">[ivy:resolve] ............ (2894kB)</span>
<span style="color: #888888;">[ivy:resolve] .. (0kB)</span>
<span style="color: #c65d09; font-weight: bold;">[ivy:resolve] [SUCCESSFUL ] org.apache.hadoop#</span>hadoop-common;2.5.2!hadoop-common.jar <span style="color: #333333;">(</span>21544ms<span style="color: #333333;">)</span>
<span style="color: #888888;">[ivy:resolve] downloading http://repo1.maven.org/maven2/org/apache/hadoop/hadoop-hdfs/2.5.2/hadoop-hdfs-2.5.2.jar ...</span>
<span style="color: #888888;">[ivy:resolve] ...................................</span>
<span style="color: #888888;">[ivy:resolve] ...................................</span>
<span style="color: #888888;">[ivy:resolve] ......................................</span>
<span style="color: #888888;">[ivy:resolve] .......................................</span>
<span style="color: #888888;">[ivy:resolve] ....................................</span>
<span style="color: #888888;">[ivy:resolve] .......................................</span>
<span style="color: #888888;">[ivy:resolve] ..........................................</span>
<span style="color: #888888;">[ivy:resolve] .......................................</span>
<span style="color: #888888;">[ivy:resolve] ..................................</span>
<span style="color: #888888;">[ivy:resolve] ........................................</span>
<span style="color: #888888;">[ivy:resolve] ..................................</span>
<span style="color: #888888;">[ivy:resolve] .........................................</span>
<span style="color: #888888;">[ivy:resolve] .............................................</span>
<span style="color: #888888;">[ivy:resolve] ...................................</span>
<span style="color: #888888;">[ivy:resolve] ......................</span>
<span style="color: #888888;">[ivy:resolve] .........................................</span>
<span style="color: #888888;">[ivy:resolve] ..........................................</span>
<span style="color: #888888;">[ivy:resolve] .............................................</span>
<span style="color: #888888;">[ivy:resolve] ........................................</span>
<span style="color: #888888;">[ivy:resolve] .....................................</span>
<span style="color: #888888;">[ivy:resolve] ............. (6928kB)</span>
<span style="color: #888888;">[ivy:resolve] .. (0kB)</span>
<span style="color: #c65d09; font-weight: bold;">[ivy:resolve] [SUCCESSFUL ] org.apache.hadoop#</span>hadoop-hdfs;2.5.2!hadoop-hdfs.jar <span style="color: #333333;">(</span>33894ms<span style="color: #333333;">)</span>
<span style="color: #888888;">[ivy:resolve] downloading http://repo1.maven.org/maven2/org/apache/hadoop/hadoop-mapreduce-client-core/2.5.2/hadoop-mapreduce-client-core-2.5.2.jar ...</span>
<span style="color: #888888;">[ivy:resolve] ....................</span>
<span style="color: #888888;">[ivy:resolve] .......................</span>
<span style="color: #888888;">[ivy:resolve] .........................</span>
<span style="color: #888888;">[ivy:resolve] ..............................</span>
<span style="color: #888888;">[ivy:resolve] ...............</span>
<span style="color: #888888;">[ivy:resolve] ...................</span>
<span style="color: #888888;">[ivy:resolve] ................. (1463kB)</span>
<span style="color: #888888;">[ivy:resolve] .. (0kB)</span>
<span style="color: #c65d09; font-weight: bold;">[ivy:resolve] [SUCCESSFUL ] org.apache.hadoop#</span>hadoop-mapreduce-client-core;2.5.2!hadoop-mapreduce-client-core.jar <span style="color: #333333;">(</span>12531ms<span style="color: #333333;">)</span>
<span style="color: #888888;">[ivy:resolve] downloading http://repo1.maven.org/maven2/org/apache/hadoop/hadoop-mapreduce-client-jobclient/2.5.2/hadoop-mapreduce-client-jobclient-2.5.2.jar ...</span>
<span style="color: #888888;">[ivy:resolve] .. (34kB)</span>
<span style="color: #888888;">[ivy:resolve] .. (0kB)</span>
<span style="color: #c65d09; font-weight: bold;">[ivy:resolve] [SUCCESSFUL ] org.apache.hadoop#</span>hadoop-mapreduce-client-jobclient;2.5.2!hadoop-mapreduce-client-jobclient.jar <span style="color: #333333;">(</span>1075ms<span style="color: #333333;">)</span>
<span style="color: #888888;">[ivy:resolve] downloading http://maven.restlet.org/org/restlet/jse/org.restlet/2.2.3/org.restlet-2.2.3.jar ...</span>
<span style="color: #888888;">[ivy:resolve] ......................</span>
<span style="color: #888888;">[ivy:resolve] ..........................</span>
<span style="color: #888888;">[ivy:resolve] ......................... (670kB)</span>
<span style="color: #888888;">[ivy:resolve] .. (0kB)</span>
<span style="color: #c65d09; font-weight: bold;">[ivy:resolve] [SUCCESSFUL ] org.restlet.jse#</span>org.restlet;2.2.3!org.restlet.jar <span style="color: #333333;">(</span>7877ms<span style="color: #333333;">)</span>
<span style="color: #888888;">[ivy:resolve] downloading http://maven.restlet.org/org/restlet/jse/org.restlet.ext.jackson/2.2.3/org.restlet.ext.jackson-2.2.3.jar ...</span>
<span style="color: #888888;">[ivy:resolve] ... (7kB)</span>
<span style="color: #888888;">[ivy:resolve] .. (0kB)</span>
<span style="color: #c65d09; font-weight: bold;">[ivy:resolve] [SUCCESSFUL ] org.restlet.jse#</span>org.restlet.ext.jackson;2.2.3!org.restlet.ext.jackson.jar <span style="color: #333333;">(</span>2971ms<span style="color: #333333;">)</span>
<span style="color: #888888;">[ivy:resolve] downloading http://maven.restlet.org/org/restlet/jse/org.restlet.ext.jaxrs/2.2.3/org.restlet.ext.jaxrs-2.2.3.jar ...</span>
<span style="color: #888888;">[ivy:resolve] ...................</span>
<span style="color: #888888;">[ivy:resolve] ............ (305kB)</span>
<span style="color: #888888;">[ivy:resolve] .. (0kB)</span>
<span style="color: #c65d09; font-weight: bold;">[ivy:resolve] [SUCCESSFUL ] org.restlet.jse#</span>org.restlet.ext.jaxrs;2.2.3!org.restlet.ext.jaxrs.jar <span style="color: #333333;">(</span>5760ms<span style="color: #333333;">)</span>
<span style="color: #888888;">[ivy:resolve] downloading http://repo1.maven.org/maven2/junit/junit/4.11/junit-4.11.jar ...</span>
<span style="color: #888888;">[ivy:resolve] ....................... (239kB)</span>
<span style="color: #888888;">[ivy:resolve] .. (0kB)</span>
<span style="color: #c65d09; font-weight: bold;">[ivy:resolve] [SUCCESSFUL ] junit#</span>junit;4.11!junit.jar <span style="color: #333333;">(</span>718ms<span style="color: #333333;">)</span>
<span style="color: #888888;">[ivy:resolve] downloading http://repo1.maven.org/maven2/org/hsqldb/hsqldb/2.2.8/hsqldb-2.2.8.jar ...</span>
<span style="color: #888888;">[ivy:resolve] ............................</span>
<span style="color: #888888;">[ivy:resolve] .........................</span>
<span style="color: #888888;">[ivy:resolve] ....................</span>
</pre>
</div>
<br />
<br />
Configure Nutch<br />
<br />
$NUTCH_HOME/runtime/local/conf/nutch-site.xml :<br />
<!-- HTML generated using hilite.me --><br />
<div style="background: #ffffff; border-width: 0.1em 0.1em 0.1em 0.8em; border: solid gray; overflow: auto; padding: 0.2em 0.6em; width: auto;">
<pre style="line-height: 125%; margin: 0;"><span style="color: #007700;"><configuration></span>
<span style="color: #007700;"><property></span>
<span style="color: #007700;"><name></span>http.agent.name<span style="color: #007700;"></name></span>
<span style="color: #007700;"><value></span>Nutty Spider<span style="color: #007700;"></value></span>
<span style="color: #007700;"></property></span>
<span style="color: #007700;"><property></span>
<span style="color: #007700;"><name></span>storage.data.store.class<span style="color: #007700;"></name></span>
<span style="color: #007700;"><value></span>org.apache.gora.hbase.store.HBaseStore<span style="color: #007700;"></value></span>
<span style="color: #007700;"><description></span>Default class for storing data<span style="color: #007700;"></description></span>
<span style="color: #007700;"></property></span>
<span style="color: #007700;"><property></span>
<span style="color: #007700;"><name></span>plugin.includes<span style="color: #007700;"></name></span> <span style="color: #007700;"><value></span>protocol-httpclient|urlfilter-regex|parse-(text|tika|js)|index-(basic|anchor)|query-(basic|site|url)|response-(json|xml)|summary-basic|scoring-opic|urlnormalizer-(pass|regex|basic)|indexer-elastic<span style="color: #007700;"></value></span>
<span style="color: #007700;"></property></span>
<span style="color: #007700;"><property></span>
<span style="color: #007700;"><name></span>db.ignore.external.links<span style="color: #007700;"></name></span>
<span style="color: #007700;"><value></span>true<span style="color: #007700;"></value></span>
<span style="color: #007700;"></property></span>
<span style="color: #007700;"><property></span>
<span style="color: #007700;"><name></span>elastic.host<span style="color: #007700;"></name></span>
<span style="color: #007700;"><value></span>10.0.2.41<span style="color: #007700;"></value></span>
<span style="color: #007700;"></property></span>
<span style="color: #007700;"><property></span>
<span style="color: #007700;"><name></span>elastic.port<span style="color: #007700;"></name></span>
<span style="color: #007700;"><value></span>9300<span style="color: #007700;"></value></span>
<span style="color: #007700;"></property></span>
<span style="color: #007700;"><property></span>
<span style="color: #007700;"><name></span>elastic.cluster<span style="color: #007700;"></name></span>
<span style="color: #007700;"><value></span>elasticsearch<span style="color: #007700;"></value></span>
<span style="color: #007700;"></property></span>
<span style="color: #007700;"><property></span>
<span style="color: #007700;"><name></span>elastic.index<span style="color: #007700;"></name></span>
<span style="color: #007700;"><value></span>nutchindex<span style="color: #007700;"></value></span>
<span style="color: #007700;"></property></span>
<span style="color: #007700;"><property></span>
<span style="color: #007700;"><name></span>parser.character.encoding.default<span style="color: #007700;"></name></span>
<span style="color: #007700;"><value></span>utf-8<span style="color: #007700;"></value></span>
<span style="color: #007700;"></property></span>
<span style="color: #007700;"><property></span>
<span style="color: #007700;"><name></span>http.content.limit<span style="color: #007700;"></name></span>
<span style="color: #007700;"><value></span>6553600<span style="color: #007700;"></value></span>
<span style="color: #007700;"></property></span>
<span style="color: #007700;"><property></span>
<span style="color: #007700;"><name></span>elastic.max.bulk.docs<span style="color: #007700;"></name></span>
<span style="color: #007700;"><value></span>250<span style="color: #007700;"></value></span>
<span style="color: #007700;"><description></span>Maximum size of the bulk in number of documents.<span style="color: #007700;"></description></span>
<span style="color: #007700;"></property></span>
<span style="color: #007700;"><property></span>
<span style="color: #007700;"><name></span>elastic.max.bulk.size<span style="color: #007700;"></name></span>
<span style="color: #007700;"><value></span>2500500<span style="color: #007700;"></value></span>
<span style="color: #007700;"><description></span>Maximum size of the bulk in bytes.<span style="color: #007700;"></description></span>
<span style="color: #007700;"></property></span>
<span style="color: #007700;"></configuration></span>
</pre>
</div>
<br />
<br />
Simple test<br />
<br />
<!-- HTML generated using hilite.me --><br />
<div style="background: #ffffff; border-width: 0.1em 0.1em 0.1em 0.8em; border: solid gray; overflow: auto; padding: 0.2em 0.6em; width: auto;">
<pre style="line-height: 125%; margin: 0;"><span style="color: #c65d09; font-weight: bold;">nutch@45883500b170:~/apache-nutch-2.3.1/runtime/local/bin$</span> ./nutch inject ~/nutch/testseed
<span style="color: #888888;">InjectorJob: starting at 2016-08-30 09:48:49</span>
<span style="color: #888888;">InjectorJob: Injecting urlDir: /home/nutch/nutch/testseed</span>
<span style="color: #888888;">InjectorJob: Using class org.apache.gora.hbase.store.HBaseStore as the Gora storage class.</span>
<span style="color: #888888;">InjectorJob: total number of urls rejected by filters: 0</span>
<span style="color: #888888;">InjectorJob: total number of urls injected after normalization and filtering: 0</span>
<span style="color: #888888;">Injector: finished at 2016-08-30 09:48:58, elapsed: 00:00:08</span>
</pre>
</div>
<br />
<br />
<h3>
Crawling the web and indexing by Elasticsearch</h3>
<br />
9300 - Elasticsearch native java port<br />
9200 - RESTful API<br />
<br />
<div style="background: #ffffff; border-width: 0.1em 0.1em 0.1em 0.8em; border: solid gray; overflow: auto; padding: 0.2em 0.6em; width: auto;">
<pre style="line-height: 125%; margin: 0;"><span style="color: #c65d09; font-weight: bold;"><pre style="color: black; font-weight: normal;"><span style="color: #c65d09; font-weight: bold; line-height: 125%;">nutch@45883500b170:~$</span><span style="line-height: 125%;"> </span>cat seed/urls.txt
<span style="color: #888888;">https://en.wikipedia.org</span></pre>
</span></pre>
<pre style="line-height: 125%; margin: 0;"><span style="color: #c65d09; font-weight: bold;">nutch@45883500b170:~$</span> nutch inject seed/urls.txt
<span style="color: #888888;">InjectorJob: starting at 2016-08-30 10:24:37</span>
<span style="color: #888888;">InjectorJob: Injecting urlDir: seed/urls.txt</span>
<span style="color: #888888;">InjectorJob: Using class org.apache.gora.hbase.store.HBaseStore as the Gora storage class.</span>
<span style="color: #888888;">InjectorJob: total number of urls rejected by filters: 0</span>
<span style="color: #888888;">InjectorJob: total number of urls injected after normalization and filtering: 1</span>
<span style="color: #888888;">Injector: finished at 2016-08-30 10:24:41, elapsed: 00:00:03</span>
<span style="color: #c65d09; font-weight: bold;">nutch@45883500b170:~$</span> nutch generate -topN 40
<span style="color: #888888;">GeneratorJob: starting at 2016-08-30 10:25:02</span>
<span style="color: #888888;">GeneratorJob: Selecting best-scoring urls due for fetch.</span>
<span style="color: #888888;">GeneratorJob: starting</span>
<span style="color: #888888;">GeneratorJob: filtering: true</span>
<span style="color: #888888;">GeneratorJob: normalizing: true</span>
<span style="color: #888888;">GeneratorJob: topN: 40</span>
<span style="color: #888888;">GeneratorJob: finished at 2016-08-30 10:25:07, time elapsed: 00:00:04</span>
<span style="color: #888888;">GeneratorJob: generated batch id: 1472552702-144817008 containing 1 URLs</span>
<span style="color: #c65d09; font-weight: bold;">nutch@45883500b170:~$</span> nutch fetch -all
<span style="color: #888888;">FetcherJob: starting at 2016-08-30 10:25:16</span>
<span style="color: #888888;">FetcherJob: fetching all</span>
<span style="color: #888888;">FetcherJob: threads: 10</span>
<span style="color: #888888;">FetcherJob: parsing: false</span>
<span style="color: #888888;">FetcherJob: resuming: false</span>
<span style="color: #888888;">FetcherJob : timelimit set for : -1</span>
<span style="color: #888888;">Using queue mode : byHost</span>
<span style="color: #888888;">Fetcher: threads: 10</span>
<span style="color: #888888;">QueueFeeder finished: total 1 records. Hit by time limit :0</span>
<span style="color: #888888;">Fetcher: throughput threshold: -1</span>
<span style="color: #888888;">Fetcher: throughput threshold sequence: 5</span>
<span style="color: #888888;">fetching https://en.wikipedia.org/ (queue crawl delay=5000ms)</span>
<span style="color: #888888;">-finishing thread FetcherThread2, activeThreads=8</span>
<span style="color: #888888;">-finishing thread FetcherThread6, activeThreads=6</span>
<span style="color: #888888;">-finishing thread FetcherThread5, activeThreads=5</span>
<span style="color: #888888;">-finishing thread FetcherThread7, activeThreads=7</span>
<span style="color: #888888;">-finishing thread FetcherThread8, activeThreads=7</span>
<span style="color: #888888;">-finishing thread FetcherThread4, activeThreads=4</span>
<span style="color: #888888;">-finishing thread FetcherThread3, activeThreads=3</span>
<span style="color: #888888;">-finishing thread FetcherThread1, activeThreads=2</span>
<span style="color: #888888;">-finishing thread FetcherThread9, activeThreads=1</span>
<span style="color: #888888;">-finishing thread FetcherThread0, activeThreads=0</span>
<span style="color: #888888;">0/0 spinwaiting/active, 1 pages, 0 errors, 0.2 0 pages/s, 0 0 kb/s, 0 URLs in 0 queues</span>
<span style="color: #888888;">-activeThreads=0</span>
<span style="color: #888888;">Using queue mode : byHost</span>
<span style="color: #888888;">Fetcher: threads: 10</span>
<span style="color: #888888;">QueueFeeder finished: total 0 records. Hit by time limit :0</span>
<span style="color: #888888;">Fetcher: throughput threshold: -1</span>
<span style="color: #888888;">Fetcher: throughput threshold sequence: 5</span>
<span style="color: #888888;">-finishing thread FetcherThread9, activeThreads=9</span>
<span style="color: #888888;">-finishing thread FetcherThread1, activeThreads=8</span>
<span style="color: #888888;">-finishing thread FetcherThread2, activeThreads=7</span>
<span style="color: #888888;">-finishing thread FetcherThread0, activeThreads=1</span>
<span style="color: #888888;">-finishing thread FetcherThread7, activeThreads=2</span>
<span style="color: #888888;">-finishing thread FetcherThread6, activeThreads=3</span>
<span style="color: #888888;">-finishing thread FetcherThread5, activeThreads=4</span>
<span style="color: #888888;">-finishing thread FetcherThread4, activeThreads=5</span>
<span style="color: #888888;">-finishing thread FetcherThread3, activeThreads=6</span>
<span style="color: #888888;">-finishing thread FetcherThread8, activeThreads=0</span>
<span style="color: #888888;">0/0 spinwaiting/active, 0 pages, 0 errors, 0.0 0 pages/s, 0 0 kb/s, 0 URLs in 0 queues</span>
<span style="color: #888888;">-activeThreads=0</span>
<span style="color: #888888;">FetcherJob: finished at 2016-08-30 10:25:32, time elapsed: 00:00:16</span>
<span style="color: #c65d09; font-weight: bold;">nutch@45883500b170:~$</span>
</pre>
</div>
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />พี่นัทhttp://www.blogger.com/profile/14291406590594822628noreply@blogger.com0tag:blogger.com,1999:blog-2488490746136830746.post-85597318739835224102016-08-30T15:24:00.001+07:002016-08-30T23:24:25.576+07:00Apache Ant & Ivy installation<br />
ANT - A build of Java Application<br />
IVY - Dependency manager<br />
<br />
<h3>
Install Ant</h3>
<div style="background: #ffffff; border-width: 0.1em 0.1em 0.1em 0.8em; border: solid gray; overflow: auto; padding: 0.2em 0.6em; width: auto;">
<pre style="line-height: 125%; margin: 0;"><span style="color: #c65d09; font-weight: bold;">nutch@45883500b170:~$</span> <span style="color: #007020;">pwd</span>
<span style="color: #888888;">/home/nutch</span>
<span style="color: #c65d09; font-weight: bold;">nutch@45883500b170:~$</span> tar xzvf /software/apache-ivy-2.4.0-bin-with-deps.tar.gz
<span style="color: #888888;">...</span>
<span style="color: #c65d09; font-weight: bold;">nutch@45883500b170:~$</span> tar xzvf /software/apache-ant-1.9.7-bin.tar.gz
<span style="color: #888888;">...</span>
<span style="color: #c65d09; font-weight: bold;">nutch@45883500b170:~$</span> ls -l
<span style="color: #888888;">total 12</span>
<span style="color: #888888;">drwxr-xr-x 6 nutch nutch 4096 Apr 9 06:38 apache-ant-1.9.7</span>
<span style="color: #888888;">drwxr-xr-x 5 nutch nutch 4096 Aug 30 07:10 apache-ivy-2.4.0</span>
<span style="color: #888888;">drwxr-xr-x 7 nutch nutch 4096 Aug 30 05:48 apache-nutch-2.3.1</span>
<span style="color: #c65d09; font-weight: bold;">nutch@45883500b170:~/apache-ant-1.9.7$</span> tail ~/.bashrc
<span style="color: #888888;"> if [ -f /usr/share/bash-completion/bash_completion ]; then</span>
<span style="color: #888888;"> . /usr/share/bash-completion/bash_completion</span>
<span style="color: #888888;"> elif [ -f /etc/bash_completion ]; then</span>
<span style="color: #888888;"> . /etc/bash_completion</span>
<span style="color: #888888;"> fi</span>
<span style="color: #888888;">fi</span>
<span style="color: #888888;">JAVA_HOME=/usr/lib/jvm/java-8-oracle</span>
<span style="color: #888888;">ANT_HOME=/home/nutch/apache-ant-1.9.7</span>
<span style="color: #888888;">PATH=$ANT_HOME/bin:$PATH</span>
<span style="color: #888888;">export JAVA_HOME ANT_HOME PATH</span>
</pre>
</div>
<br />
<br />
<!-- HTML generated using hilite.me --><br />
<div style="background: #ffffff; border-width: 0.1em 0.1em 0.1em 0.8em; border: solid gray; overflow: auto; padding: 0.2em 0.6em; width: auto;">
<pre style="line-height: 125%; margin: 0;"><span style="color: #c65d09; font-weight: bold;">nutch@45883500b170:/$</span> <span style="color: #007020;">cd</span>
<span style="color: #c65d09; font-weight: bold;">nutch@45883500b170:~$</span><span style="color: #007020;"> </span>
<span style="color: #c65d09; font-weight: bold;">nutch@45883500b170:~$</span><span style="color: #007020;"> cd </span>apache-ant-1.9.7/
<span style="color: #c65d09; font-weight: bold;">nutch@45883500b170:~/apache-ant-1.9.7$</span>
<span style="color: #c65d09; font-weight: bold;">nutch@45883500b170:~/apache-ant-1.9.7$</span>
<span style="color: #c65d09; font-weight: bold;">nutch@45883500b170:~/apache-ant-1.9.7$</span> ant -f fetch.xml -Ddest<span style="color: #333333;">=</span>system
<span style="color: #888888;">Buildfile: /home/nutch/apache-ant-1.9.7/fetch.xml</span>
<span style="color: #888888;">pick-dest:</span>
<span style="color: #888888;"> [echo] Downloading to /home/nutch/apache-ant-1.9.7/lib</span>
<span style="color: #888888;">probe-m2:</span>
<span style="color: #888888;">download-m2:</span>
<span style="color: #888888;"> [echo] Downloading to /home/nutch/apache-ant-1.9.7/lib</span>
<span style="color: #888888;"> [get] Getting: http://repo1.maven.org/maven2/org/apache/maven/maven-artifact-ant/2.0.4/maven-artifact-ant-2.0.4-dep.jar</span>
<span style="color: #888888;"> [get] To: /home/nutch/apache-ant-1.9.7/lib/maven-artifact-ant-2.0.4-dep.jar</span>
<span style="color: #888888;"> [get] ....................................................</span>
<span style="color: #888888;"> [get] ...................................</span>
<span style="color: #888888;">dont-validate-m2-checksum:</span>
<span style="color: #888888;">validate-m2-checksum:</span>
<span style="color: #888888;">checksum-mismatch:</span>
<span style="color: #888888;">checksum-match:</span>
<span style="color: #888888;">get-m2:</span>
<span style="color: #888888;">macros:</span>
<span style="color: #888888;">init:</span>
<span style="color: #888888;">logging:</span>
<span style="color: #888888;">[artifact:dependencies] Downloading: log4j/log4j/1.2.14/log4j-1.2.14.pom</span>
<span style="color: #888888;">[artifact:dependencies] Transferring 2K</span>
<span style="color: #888888;">[artifact:dependencies] Downloading: log4j/log4j/1.2.14/log4j-1.2.14.jar</span>
<span style="color: #888888;">[artifact:dependencies] Transferring 358K</span>
<span style="color: #888888;"> [copy] Copying 1 file to /home/nutch/apache-ant-1.9.7/lib</span>
<span style="color: #888888;">[artifact:dependencies] Downloading: commons-logging/commons-logging-api/1.1/commons-logging-api-1.1.pom</span>
<span style="color: #888888;">[artifact:dependencies] Transferring 5K</span>
<span style="color: #888888;">[artifact:dependencies] Downloading: commons-logging/commons-logging-api/1.1/commons-logging-api-1.1.jar</span>
<span style="color: #888888;">[artifact:dependencies] Transferring 43K</span>
<span style="color: #888888;"> [copy] Copying 1 file to /home/nutch/apache-ant-1.9.7/lib</span>
<span style="color: #888888;">junit:</span>
<span style="color: #888888;">[artifact:dependencies] Downloading: junit/junit/4.11/junit-4.11.pom</span>
<span style="color: #888888;">[artifact:dependencies] Transferring 2K</span>
<span style="color: #888888;">[artifact:dependencies] Downloading: org/hamcrest/hamcrest-core/1.3/hamcrest-core-1.3.pom</span>
<span style="color: #888888;">[artifact:dependencies] Transferring 0K</span>
<span style="color: #888888;">[artifact:dependencies] Downloading: org/hamcrest/hamcrest-parent/1.3/hamcrest-parent-1.3.pom</span>
<span style="color: #888888;">[artifact:dependencies] Transferring 1K</span>
<span style="color: #888888;">[artifact:dependencies] Downloading: org/hamcrest/hamcrest-core/1.3/hamcrest-core-1.3.jar</span>
<span style="color: #888888;">[artifact:dependencies] Transferring 43K</span>
<span style="color: #888888;">[artifact:dependencies] Downloading: junit/junit/4.11/junit-4.11.jar</span>
<span style="color: #888888;">[artifact:dependencies] Transferring 239K</span>
<span style="color: #888888;"> [copy] Copying 2 files to /home/nutch/apache-ant-1.9.7/lib</span>
<span style="color: #888888;">xml:</span>
<span style="color: #888888;">[artifact:dependencies] Downloading: xalan/xalan/2.7.1/xalan-2.7.1.pom</span>
<span style="color: #888888;">[artifact:dependencies] Transferring 1K</span>
<span style="color: #888888;">[artifact:dependencies] Downloading: org/apache/apache/4/apache-4.pom</span>
<span style="color: #888888;">[artifact:dependencies] Transferring 4K</span>
<span style="color: #888888;">[artifact:dependencies] Downloading: xalan/serializer/2.7.1/serializer-2.7.1.pom</span>
<span style="color: #888888;">[artifact:dependencies] Transferring 1K</span>
<span style="color: #888888;">[artifact:dependencies] Downloading: xml-apis/xml-apis/1.3.04/xml-apis-1.3.04.pom</span>
<span style="color: #888888;">[artifact:dependencies] Transferring 1K</span>
<span style="color: #888888;">[artifact:dependencies] Downloading: org/apache/apache/3/apache-3.pom</span>
<span style="color: #888888;">[artifact:dependencies] Transferring 3K</span>
<span style="color: #888888;">[artifact:dependencies] Downloading: xalan/serializer/2.7.1/serializer-2.7.1.jar</span>
<span style="color: #888888;">[artifact:dependencies] Transferring 271K</span>
<span style="color: #888888;">[artifact:dependencies] Downloading: xml-apis/xml-apis/1.3.04/xml-apis-1.3.04.jar</span>
<span style="color: #888888;">[artifact:dependencies] Transferring 189K</span>
<span style="color: #888888;">[artifact:dependencies] Downloading: xalan/xalan/2.7.1/xalan-2.7.1.jar</span>
<span style="color: #888888;">[artifact:dependencies] Transferring 3101K</span>
<span style="color: #888888;"> [copy] Copying 3 files to /home/nutch/apache-ant-1.9.7/lib</span>
<span style="color: #888888;">[artifact:dependencies] Downloading: xml-resolver/xml-resolver/1.2/xml-resolver-1.2.pom</span>
<span style="color: #888888;">[artifact:dependencies] Transferring 1K</span>
<span style="color: #888888;">[artifact:dependencies] Downloading: xml-resolver/xml-resolver/1.2/xml-resolver-1.2.jar</span>
<span style="color: #888888;">[artifact:dependencies] Transferring 82K</span>
<span style="color: #888888;"> [copy] Copying 1 file to /home/nutch/apache-ant-1.9.7/lib</span>
<span style="color: #888888;">networking:</span>
<span style="color: #888888;">[artifact:dependencies] Downloading: commons-net/commons-net/1.4.1/commons-net-1.4.1.pom</span>
<span style="color: #888888;">[artifact:dependencies] Transferring 4K</span>
<span style="color: #888888;">[artifact:dependencies] Downloading: oro/oro/2.0.8/oro-2.0.8.pom</span>
<span style="color: #888888;">[artifact:dependencies] Transferring 0K</span>
<span style="color: #888888;">[artifact:dependencies] Downloading: oro/oro/2.0.8/oro-2.0.8.jar</span>
<span style="color: #888888;">[artifact:dependencies] Transferring 63K</span>
<span style="color: #888888;">[artifact:dependencies] Downloading: commons-net/commons-net/1.4.1/commons-net-1.4.1.jar</span>
<span style="color: #888888;">[artifact:dependencies] Transferring 176K</span>
<span style="color: #888888;"> [copy] Copying 2 files to /home/nutch/apache-ant-1.9.7/lib</span>
<span style="color: #888888;">[artifact:dependencies] Downloading: com/jcraft/jsch/0.1.50/jsch-0.1.50.pom</span>
<span style="color: #888888;">[artifact:dependencies] Transferring 3K</span>
<span style="color: #888888;">[artifact:dependencies] Downloading: org/sonatype/oss/oss-parent/6/oss-parent-6.pom</span>
<span style="color: #888888;">[artifact:dependencies] Transferring 4K</span>
<span style="color: #888888;">[artifact:dependencies] Downloading: com/jcraft/jsch/0.1.50/jsch-0.1.50.jar</span>
<span style="color: #888888;">[artifact:dependencies] Transferring 248K</span>
<span style="color: #888888;"> [copy] Copying 1 file to /home/nutch/apache-ant-1.9.7/lib</span>
<span style="color: #888888;">regexp:</span>
<span style="color: #888888;">[artifact:dependencies] Downloading: regexp/regexp/1.3/regexp-1.3.pom</span>
<span style="color: #888888;">[artifact:dependencies] Transferring 0K</span>
<span style="color: #888888;">[artifact:dependencies] Downloading: regexp/regexp/1.3/regexp-1.3.jar</span>
<span style="color: #888888;">[artifact:dependencies] Transferring 24K</span>
<span style="color: #888888;"> [copy] Copying 1 file to /home/nutch/apache-ant-1.9.7/lib</span>
<span style="color: #888888;">antlr:</span>
<span style="color: #888888;">[artifact:dependencies] Downloading: antlr/antlr/2.7.7/antlr-2.7.7.pom</span>
<span style="color: #888888;">[artifact:dependencies] Transferring 0K</span>
<span style="color: #888888;">[artifact:dependencies] Downloading: antlr/antlr/2.7.7/antlr-2.7.7.jar</span>
<span style="color: #888888;">[artifact:dependencies] Transferring 434K</span>
<span style="color: #888888;"> [copy] Copying 1 file to /home/nutch/apache-ant-1.9.7/lib</span>
<span style="color: #888888;">bcel:</span>
<span style="color: #888888;">[artifact:dependencies] Downloading: bcel/bcel/5.1/bcel-5.1.pom</span>
<span style="color: #888888;">[artifact:dependencies] Transferring 0K</span>
<span style="color: #888888;">[artifact:dependencies] Downloading: regexp/regexp/1.2/regexp-1.2.pom</span>
<span style="color: #888888;">[artifact:dependencies] Transferring 0K</span>
<span style="color: #888888;">[artifact:dependencies] Downloading: regexp/regexp/1.2/regexp-1.2.jar</span>
<span style="color: #888888;">[artifact:dependencies] Transferring 29K</span>
<span style="color: #888888;">[artifact:dependencies] Downloading: bcel/bcel/5.1/bcel-5.1.jar</span>
<span style="color: #888888;">[artifact:dependencies] Transferring 503K</span>
<span style="color: #888888;"> [copy] Copying 2 files to /home/nutch/apache-ant-1.9.7/lib</span>
<span style="color: #888888;">jdepend:</span>
<span style="color: #888888;">[artifact:dependencies] Downloading: jdepend/jdepend/2.9.1/jdepend-2.9.1.pom</span>
<span style="color: #888888;">[artifact:dependencies] Transferring 0K</span>
<span style="color: #888888;">[artifact:dependencies] Downloading: jdepend/jdepend/2.9.1/jdepend-2.9.1.jar</span>
<span style="color: #888888;">[artifact:dependencies] Transferring 56K</span>
<span style="color: #888888;"> [copy] Copying 1 file to /home/nutch/apache-ant-1.9.7/lib</span>
<span style="color: #888888;">bsf:</span>
<span style="color: #888888;">[artifact:dependencies] Downloading: bsf/bsf/2.4.0/bsf-2.4.0.pom</span>
<span style="color: #888888;">[artifact:dependencies] Transferring 1K</span>
<span style="color: #888888;">[artifact:dependencies] Downloading: commons-logging/commons-logging/1.0.4/commons-logging-1.0.4.pom</span>
<span style="color: #888888;">[artifact:dependencies] Transferring 5K</span>
<span style="color: #888888;">[artifact:dependencies] Downloading: bsf/bsf/2.4.0/bsf-2.4.0.jar</span>
<span style="color: #888888;">[artifact:dependencies] Transferring 110K</span>
<span style="color: #888888;">[artifact:dependencies] Downloading: commons-logging/commons-logging/1.0.4/commons-logging-1.0.4.jar</span>
<span style="color: #888888;">[artifact:dependencies] Transferring 37K</span>
<span style="color: #888888;"> [copy] Copying 2 files to /home/nutch/apache-ant-1.9.7/lib</span>
<span style="color: #888888;">debugging:</span>
<span style="color: #888888;">[artifact:dependencies] Downloading: which/which/1.0/which-1.0.pom</span>
<span style="color: #888888;">[artifact:dependencies] Transferring 0K</span>
<span style="color: #888888;">[artifact:dependencies] Downloading: which/which/1.0/which-1.0.jar</span>
<span style="color: #888888;">[artifact:dependencies] Transferring 16K</span>
<span style="color: #888888;"> [copy] Copying 1 file to /home/nutch/apache-ant-1.9.7/lib</span>
<span style="color: #888888;">jruby:</span>
<span style="color: #888888;">[artifact:dependencies] Downloading: org/jruby/jruby/0.9.8/jruby-0.9.8.pom</span>
<span style="color: #888888;">[artifact:dependencies] Transferring 5K</span>
<span style="color: #888888;">[artifact:dependencies] Downloading: org/jruby/shared/0.9.8/shared-0.9.8.pom</span>
<span style="color: #888888;">[artifact:dependencies] Transferring 4K</span>
<span style="color: #888888;">[artifact:dependencies] Downloading: asm/asm-commons/2.2.3/asm-commons-2.2.3.pom</span>
<span style="color: #888888;">[artifact:dependencies] Transferring 0K</span>
<span style="color: #888888;">[artifact:dependencies] Downloading: asm/asm-parent/2.2.3/asm-parent-2.2.3.pom</span>
<span style="color: #888888;">[artifact:dependencies] Transferring 2K</span>
<span style="color: #888888;">[artifact:dependencies] Downloading: asm/asm-tree/2.2.3/asm-tree-2.2.3.pom</span>
<span style="color: #888888;">[artifact:dependencies] Transferring 0K</span>
<span style="color: #888888;">[artifact:dependencies] Downloading: asm/asm/2.2.3/asm-2.2.3.pom</span>
<span style="color: #888888;">[artifact:dependencies] Transferring 0K</span>
<span style="color: #888888;">[artifact:dependencies] Downloading: asm/asm/2.2.3/asm-2.2.3.jar</span>
<span style="color: #888888;">[artifact:dependencies] Transferring 34K</span>
<span style="color: #888888;">[artifact:dependencies] Downloading: asm/asm-tree/2.2.3/asm-tree-2.2.3.jar</span>
<span style="color: #888888;">[artifact:dependencies] Transferring 15K</span>
<span style="color: #888888;">[artifact:dependencies] Downloading: org/jruby/jruby/0.9.8/jruby-0.9.8.jar</span>
<span style="color: #888888;">[artifact:dependencies] Transferring 1644K</span>
<span style="color: #888888;">[artifact:dependencies] Downloading: asm/asm-commons/2.2.3/asm-commons-2.2.3.jar</span>
<span style="color: #888888;">[artifact:dependencies] Transferring 14K</span>
<span style="color: #888888;"> [copy] Copying 4 files to /home/nutch/apache-ant-1.9.7/lib</span>
<span style="color: #888888;">beanshell:</span>
<span style="color: #888888;">[artifact:dependencies] Downloading: org/beanshell/bsh/2.0b4/bsh-2.0b4.pom</span>
<span style="color: #888888;">[artifact:dependencies] Transferring 1K</span>
<span style="color: #888888;">[artifact:dependencies] Downloading: org/beanshell/beanshell/2.0b4/beanshell-2.0b4.pom</span>
<span style="color: #888888;">[artifact:dependencies] Transferring 1K</span>
<span style="color: #888888;">[artifact:dependencies] Downloading: org/beanshell/bsh/2.0b4/bsh-2.0b4.jar</span>
<span style="color: #888888;">[artifact:dependencies] Transferring 275K</span>
<span style="color: #888888;"> [copy] Copying 1 file to /home/nutch/apache-ant-1.9.7/lib</span>
<span style="color: #888888;">[artifact:dependencies] Downloading: org/beanshell/bsh-core/2.0b4/bsh-core-2.0b4.pom</span>
<span style="color: #888888;">[artifact:dependencies] Transferring 0K</span>
<span style="color: #888888;">[artifact:dependencies] Downloading: org/beanshell/bsh-core/2.0b4/bsh-core-2.0b4.jar</span>
<span style="color: #888888;">[artifact:dependencies] Transferring 140K</span>
<span style="color: #888888;"> [copy] Copying 1 file to /home/nutch/apache-ant-1.9.7/lib</span>
<span style="color: #888888;">rhino:</span>
<span style="color: #888888;">[artifact:dependencies] Downloading: rhino/js/1.6R7/js-1.6R7.pom</span>
<span style="color: #888888;">[artifact:dependencies] Transferring 0K</span>
<span style="color: #888888;">[artifact:dependencies] Downloading: rhino/js/1.6R7/js-1.6R7.jar</span>
<span style="color: #888888;">[artifact:dependencies] Transferring 794K</span>
<span style="color: #888888;"> [copy] Copying 1 file to /home/nutch/apache-ant-1.9.7/lib</span>
<span style="color: #888888;">script:</span>
<span style="color: #888888;">javamail:</span>
<span style="color: #888888;">[artifact:dependencies] Downloading: javax/mail/mail/1.4/mail-1.4.pom</span>
<span style="color: #888888;">[artifact:dependencies] Transferring 0K</span>
<span style="color: #888888;">[artifact:dependencies] Downloading: javax/activation/activation/1.1/activation-1.1.pom</span>
<span style="color: #888888;">[artifact:dependencies] Transferring 1K</span>
<span style="color: #888888;">[artifact:dependencies] Downloading: javax/mail/mail/1.4/mail-1.4.jar</span>
<span style="color: #888888;">[artifact:dependencies] Transferring 379K</span>
<span style="color: #888888;">[artifact:dependencies] Downloading: javax/activation/activation/1.1/activation-1.1.jar</span>
<span style="color: #888888;">[artifact:dependencies] Transferring 61K</span>
<span style="color: #888888;"> [copy] Copying 2 files to /home/nutch/apache-ant-1.9.7/lib</span>
<span style="color: #888888;">jspc:</span>
<span style="color: #888888;">[artifact:dependencies] Downloading: tomcat/jasper-compiler/4.1.36/jasper-compiler-4.1.36.pom</span>
<span style="color: #888888;">[artifact:dependencies] [WARNING] Unable to get resource from repository remote (http://repo1.maven.org/maven2/)</span>
<span style="color: #888888;">[artifact:dependencies] Downloading: tomcat/jasper-compiler/4.1.36/jasper-compiler-4.1.36.pom</span>
<span style="color: #888888;">[artifact:dependencies] [WARNING] Unable to get resource from repository central (http://repo1.maven.org/maven2)</span>
<span style="color: #888888;">[artifact:dependencies] Downloading: tomcat/jasper-compiler/4.1.36/jasper-compiler-4.1.36.jar</span>
<span style="color: #888888;">[artifact:dependencies] Transferring 179K</span>
<span style="color: #888888;"> [copy] Copying 1 file to /home/nutch/apache-ant-1.9.7/lib</span>
<span style="color: #888888;">[artifact:dependencies] Downloading: tomcat/jasper-runtime/4.1.36/jasper-runtime-4.1.36.pom</span>
<span style="color: #888888;">[artifact:dependencies] [WARNING] Unable to get resource from repository remote (http://repo1.maven.org/maven2/)</span>
<span style="color: #888888;">[artifact:dependencies] Downloading: tomcat/jasper-runtime/4.1.36/jasper-runtime-4.1.36.pom</span>
<span style="color: #888888;">[artifact:dependencies] [WARNING] Unable to get resource from repository central (http://repo1.maven.org/maven2)</span>
<span style="color: #888888;">[artifact:dependencies] Downloading: tomcat/jasper-runtime/4.1.36/jasper-runtime-4.1.36.jar</span>
<span style="color: #888888;">[artifact:dependencies] Transferring 70K</span>
<span style="color: #888888;"> [copy] Copying 1 file to /home/nutch/apache-ant-1.9.7/lib</span>
<span style="color: #888888;">[artifact:dependencies] Downloading: javax/servlet/servlet-api/2.3/servlet-api-2.3.pom</span>
<span style="color: #888888;">[artifact:dependencies] Transferring 0K</span>
<span style="color: #888888;">[artifact:dependencies] Downloading: javax/servlet/servlet-api/2.3/servlet-api-2.3.jar</span>
<span style="color: #888888;">[artifact:dependencies] Transferring 76K</span>
<span style="color: #888888;"> [copy] Copying 1 file to /home/nutch/apache-ant-1.9.7/lib</span>
<span style="color: #888888;">jai:</span>
<span style="color: #888888;">[artifact:dependencies] Downloading: javax/media/jai-core/1.1.3/jai-core-1.1.3.pom</span>
<span style="color: #888888;">[artifact:dependencies] Transferring 1K</span>
<span style="color: #888888;">[artifact:dependencies] Downloading: javax/media/jai-core/1.1.3/jai-core-1.1.3.jar</span>
<span style="color: #888888;">[artifact:dependencies] Transferring 1856K</span>
<span style="color: #888888;"> [copy] Copying 1 file to /home/nutch/apache-ant-1.9.7/lib</span>
<span style="color: #888888;">[artifact:dependencies] Downloading: com/sun/media/jai-codec/1.1.3/jai-codec-1.1.3.pom</span>
<span style="color: #888888;">[artifact:dependencies] Transferring 1K</span>
<span style="color: #888888;">[artifact:dependencies] Downloading: com/sun/media/jai-codec/1.1.3/jai-codec-1.1.3.jar</span>
<span style="color: #888888;">[artifact:dependencies] Transferring 252K</span>
<span style="color: #888888;"> [copy] Copying 1 file to /home/nutch/apache-ant-1.9.7/lib</span>
<span style="color: #888888;">nonm2-macros:</span>
<span style="color: #888888;">init-no-m2:</span>
<span style="color: #888888;">init-cache:</span>
<span style="color: #888888;">-setup-temp-cache:</span>
<span style="color: #888888;"> [mkdir] Created dir: /home/nutch/.ant/tempcache</span>
<span style="color: #888888;">-fetch-netrexx:</span>
<span style="color: #888888;">-fetch-netrexx-no-commons-net:</span>
<span style="color: #888888;"> [get] Getting: ftp://ftp.software.ibm.com/software/awdtools/netrexx/NetRexx.zip</span>
<span style="color: #888888;"> [get] To: /home/nutch/.ant/tempcache/NetRexx.zip</span>
<span style="color: #888888;"> [get] Error getting ftp://ftp.software.ibm.com/software/awdtools/netrexx/NetRexx.zip to /home/nutch/.ant/tempcache/NetRexx.zip</span>
<span style="color: red;"><span style="color: #888888;">BUILD FAILED</span>
<span style="color: #888888;">/home/nutch/apache-ant-1.9.7/fetch.xml:328: java.net.ConnectException: Connection timed out</span>
<span style="color: #888888;"> at java.net.PlainSocketImpl.socketConnect(Native Method)</span>
<span style="color: #888888;"> at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)</span>
<span style="color: #888888;"> at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)</span>
<span style="color: #888888;"> at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)</span>
<span style="color: #888888;"> at java.net.Socket.connect(Socket.java:589)</span>
<span style="color: #888888;"> at java.net.Socket.connect(Socket.java:538)</span>
<span style="color: #888888;"> at sun.net.ftp.impl.FtpClient.doConnect(FtpClient.java:957)</span>
<span style="color: #888888;"> at sun.net.ftp.impl.FtpClient.tryConnect(FtpClient.java:917)</span>
<span style="color: #888888;"> at sun.net.ftp.impl.FtpClient.connect(FtpClient.java:1012)</span>
<span style="color: #888888;"> at sun.net.ftp.impl.FtpClient.connect(FtpClient.java:998)</span>
<span style="color: #888888;"> at sun.net.www.protocol.ftp.FtpURLConnection.connect(FtpURLConnection.java:294)</span>
<span style="color: #888888;"> at org.apache.tools.ant.taskdefs.Get$GetThread.openConnection(Get.java:728)</span>
<span style="color: #888888;"> at org.apache.tools.ant.taskdefs.Get$GetThread.get(Get.java:641)</span>
<span style="color: #888888;"> at org.apache.tools.ant.taskdefs.Get$GetThread.run(Get.java:631)</span>
<span style="color: #888888;">Total time: 4 minutes 17 seconds</span></span>
</pre>
</div>
<span style="color: red;">*** File not found in that location, so use an alternative location <a href="http://www.netrexx.org/downloads.nsp" target="_blank">here</a> . And modify fetch.xml to bypass download files again in next run.</span><br />
<br />
<h3>
Manual install Ivy</h3>
<!-- HTML generated using hilite.me --><br />
<div style="background: #ffffff; border-width: 0.1em 0.1em 0.1em 0.8em; border: solid gray; overflow: auto; padding: 0.2em 0.6em; width: auto;">
<pre style="line-height: 125%; margin: 0;"><span style="color: #c65d09; font-weight: bold;">nutch@45883500b170:~/apache-ant-1.9.7/lib$</span> cp -rp ~/apache-ivy-2.4.0/ivy-2.4.0.jar .
<span style="color: #c65d09; font-weight: bold;">nutch@45883500b170:~/apache-ant-1.9.7/lib$</span> </pre>
</div>
<br />
<br />
<br />
<br />
<br />พี่นัทhttp://www.blogger.com/profile/14291406590594822628noreply@blogger.com0tag:blogger.com,1999:blog-2488490746136830746.post-24470366258129850032016-08-28T20:16:00.000+07:002016-08-29T14:06:13.951+07:00Apache HBase - Pseudo-distributed (Part II)<br />
Pseudo-distributed mode means that HBase still runs completely on a single host, but each HBase daemon (HMaster, HRegionServer, and ZooKeeper) runs as a separate process<br />
<br />
<br />
<h3>
<br />Prepare HDFS and <a href="https://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-hdfs/HdfsPermissionsGuide.html" target="_blank">permission</a> of directory</h3>
<!-- HTML generated using hilite.me --><br />
<div style="background: #ffffff; border-width: 0.1em 0.1em 0.1em 0.8em; border: solid gray; overflow: auto; padding: 0.2em 0.6em; width: auto;">
<pre style="line-height: 125%; margin: 0;"><span style="color: #c65d09; font-weight: bold;">hadoop@45883500b170:~/hadoop-2.7.3$</span> bin/hdfs dfs -mkdir /user/hbase
<span style="color: #c65d09; font-weight: bold;">hadoop@45883500b170:~/hadoop-2.7.3/bin$</span> ./hdfs dfs -chown -R hbase:hbase /user/hbase
<span style="color: #c65d09; font-weight: bold;">hadoop@45883500b170:~/hadoop-2.7.3/bin$</span> ./hdfs dfs -ls /user
<span style="color: #888888;">Found 2 items</span>
<span style="color: #888888;">drwxr-xr-x - hadoop supergroup 0 2016-08-28 12:27 /user/hadoop</span>
<span style="color: #888888;">drwxr-xr-x - hbase hbase 0 2016-08-29 06:20 /user/hbase</span>
</pre>
</div>
<br />
<br />
<h3>
Re-configuration HBase</h3>
<br />
hbase-site.xml:<br />
<!-- HTML generated using hilite.me --><br />
<div style="background: #ffffff; border-width: 0.1em 0.1em 0.1em 0.8em; border: solid gray; overflow: auto; padding: 0.2em 0.6em; width: auto;">
<pre style="line-height: 125%; margin: 0;"><span style="color: #007700;"><configuration></span>
<span style="color: #007700;"><property></span>
<span style="color: #007700;"><name></span>hbase.rootdir<span style="color: #007700;"></name></span>
<span style="color: #888888;"><!-- value>file:///home/hbase/hbase-1.2.2</value --></span>
<span style="color: #007700;"><value></span>hdfs://localhost:9000/user/hbase<span style="color: #007700;"></value></span>
<span style="color: #007700;"></property></span>
<span style="color: #007700;"><property></span>
<span style="color: #007700;"><name></span>hbase.zookeeper.property.dataDir<span style="color: #007700;"></name></span>
<span style="color: #007700;"><value></span>/home/hbase/zookeeper<span style="color: #007700;"></value></span>
<span style="color: #007700;"></property></span>
<span style="color: #007700;"><property></span>
<span style="color: #007700;"><name></span>hbase.cluster.distributed<span style="color: #007700;"></name></span>
<span style="color: #007700;"><value></span>true<span style="color: #007700;"></value></span>
<span style="color: #007700;"></property></span>
<span style="color: #007700;"></configuration></span>
</pre>
</div>
<br />
hbase-env.sh:<br />
...<br />
<span style="color: blue;">export JAVA_HOME=/usr/lib/jvm/java-8-oracle</span><br />
...<br />
<br />
<br />
SSH bypass a passphrase<br />
<br />
<!-- HTML generated using hilite.me --><br />
<div style="background: #ffffff; border-width: 0.1em 0.1em 0.1em 0.8em; border: solid gray; overflow: auto; padding: 0.2em 0.6em; width: auto;">
<pre style="line-height: 125%; margin: 0;"><span style="color: #c65d09; font-weight: bold;">hbase@45883500b170:~/hbase-1.2.2/bin$</span> ssh-keygen -t rsa -P <span style="background-color: #fff0f0;">''</span> -f ~/.ssh/id_rsa
<span style="color: #888888;">Generating public/private rsa key pair.</span>
<span style="color: #888888;">Your identification has been saved in /home/hbase/.ssh/id_rsa.</span>
<span style="color: #888888;">Your public key has been saved in /home/hbase/.ssh/id_rsa.pub.</span>
<span style="color: #888888;">The key fingerprint is:</span>
<span style="color: #888888;">57:57:7d:53:3d:54:3e:c5:fe:ef:58:db:1e:07:c1:51 hbase@45883500b170</span>
<span style="color: #888888;">The key's randomart image is:</span>
<span style="color: #888888;">+--[ RSA 2048]----+</span>
<span style="color: #888888;">| o+E|</span>
<span style="color: #888888;">| . **|</span>
<span style="color: #888888;">| . +o=|</span>
<span style="color: #888888;">| . . .o|</span>
<span style="color: #888888;">| S . . .|</span>
<span style="color: #888888;">| . ..|</span>
<span style="color: #888888;">| .+|</span>
<span style="color: #888888;">| o*|</span>
<span style="color: #888888;">| .++|</span>
<span style="color: #888888;">+-----------------+</span>
<span style="color: #c65d09; font-weight: bold;">hbase@45883500b170:~/hbase-1.2.2/bin$</span> cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys
<span style="color: #c65d09; font-weight: bold;">hbase@45883500b170:~/hbase-1.2.2/bin$</span> chmod 0600 ~/.ssh/authorized_keys
</pre>
</div>
<br />
<br />
<br />
Starting up HBase<br />
<!-- HTML generated using hilite.me --><br />
<div style="background: #ffffff; border-width: 0.1em 0.1em 0.1em 0.8em; border: solid gray; overflow: auto; padding: 0.2em 0.6em; width: auto;">
<pre style="line-height: 125%; margin: 0;"><span style="color: #c65d09; font-weight: bold;">hbase@45883500b170:~/hbase-1.2.2/bin$</span> ./start-hbase.sh
<span style="color: #888888;">localhost: starting zookeeper, logging to /home/hbase/hbase-1.2.2/bin/../logs/hbase-hbase-zookeeper-45883500b170.out</span>
<span style="color: #888888;">starting master, logging to /home/hbase/hbase-1.2.2/bin/../logs/hbase--master-45883500b170.out</span>
<span style="color: #888888;">Java HotSpot(TM) 64-Bit Server VM warning: ignoring option PermSize=128m; support was removed in 8.0</span>
<span style="color: #888888;">Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=128m; support was removed in 8.0</span>
<span style="color: #888888;">starting regionserver, logging to /home/hbase/hbase-1.2.2/bin/../logs/hbase--1-regionserver-45883500b170.out</span>
</pre>
</div>
<br />
<br />
Whiling start up HBase it creates required files and directory in HDFS as below<br />
<br />
<br />
Open and Hadoop Web Interface<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjQR7piZwmmGDWWRfnhuhr_uUahHdqMFUpPIisHF0ESWpl1AFc8l2WlyIBizY7N-BSBFbz-wgCNsg0zmOtVZD6mgwUfDnmGYeFflZQcQlLw5Jk1g3cxEeuGW51kIhrSp6O5cMZgQiEebejT/s1600/Selection_028.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="246" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjQR7piZwmmGDWWRfnhuhr_uUahHdqMFUpPIisHF0ESWpl1AFc8l2WlyIBizY7N-BSBFbz-wgCNsg0zmOtVZD6mgwUfDnmGYeFflZQcQlLw5Jk1g3cxEeuGW51kIhrSp6O5cMZgQiEebejT/s640/Selection_028.png" width="640" /></a></div>
<br />
Testing<br />
<br />
<!-- HTML generated using hilite.me --><br />
<div style="background: #ffffff; border-width: 0.1em 0.1em 0.1em 0.8em; border: solid gray; overflow: auto; padding: 0.2em 0.6em; width: auto;">
<pre style="line-height: 125%; margin: 0;"><span style="color: #c65d09; font-weight: bold;">hbase@45883500b170:~/hbase-1.2.2/bin$</span> ./hbase shell
<span style="color: #888888;">2016-08-29 06:47:10,665 WARN [main] util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable</span>
<span style="color: #888888;">HBase Shell; enter 'help<RETURN>' for list of supported commands.</span>
<span style="color: #888888;">Type "exit<RETURN>" to leave the HBase Shell</span>
<span style="color: #888888;">Version 1.2.2, r3f671c1ead70d249ea4598f1bbcc5151322b3a13, Fri Jul 1 08:28:55 CDT 2016</span>
<span style="color: #888888;">hbase(main):001:0> create 'test', 'cf'</span>
<span style="color: #888888;">0 row(s) in 4.8240 seconds</span>
<span style="color: #888888;">=> Hbase::Table - test</span>
<span style="color: #888888;">hbase(main):002:0> list 'test'</span>
<span style="color: #888888;">TABLE </span>
<span style="color: #888888;">test </span>
<span style="color: #888888;">1 row(s) in 0.0280 seconds</span>
<span style="color: #888888;">=> ["test"]</span>
<span style="color: #888888;">hbase(main):003:0> put 'test', 'row1', 'cf:a', 'value1'</span>
<span style="color: #888888;">0 row(s) in 0.1310 seconds</span>
<span style="color: #888888;">hbase(main):004:0> put 'test', 'row2', 'cf:b', 'value2'</span>
<span style="color: #888888;">0 row(s) in 0.0200 seconds</span>
<span style="color: #888888;">hbase(main):005:0> put 'test', 'row3', 'cf:c', 'value3'</span>
<span style="color: #888888;">0 row(s) in 0.0150 seconds</span>
<span style="color: #888888;">hbase(main):006:0> scan 'test'</span>
<span style="color: #888888;">ROW COLUMN+CELL </span>
<span style="color: #888888;"> row1 column=cf:a, timestamp=1472453260190, value=value1 </span>
<span style="color: #888888;"> row2 column=cf:b, timestamp=1472453264719, value=value2 </span>
<span style="color: #888888;"> row3 column=cf:c, timestamp=1472453270502, value=value3 </span>
<span style="color: #888888;">3 row(s) in 0.0350 seconds</span>
<span style="color: #888888;">hbase(main):007:0> get 'test', 'row1'</span>
<span style="color: #888888;">COLUMN CELL </span>
<span style="color: #888888;"> cf:a timestamp=1472453260190, value=value1 </span>
<span style="color: #888888;">1 row(s) in 0.0350 seconds</span>
<span style="color: #888888;">hbase(main):008:0> exit</span>
</pre>
</div>
<br />
<br />พี่นัทhttp://www.blogger.com/profile/14291406590594822628noreply@blogger.com0tag:blogger.com,1999:blog-2488490746136830746.post-66569616408480466892016-08-28T20:14:00.001+07:002016-08-28T20:18:36.500+07:00Apache Hadoop - Pseudo-Distributed<br />
<br />
<div style="background: #ffffff; border-width: 0.1em 0.1em 0.1em 0.8em; border: solid gray; overflow: auto; padding: 0.2em 0.6em; width: auto;">
<pre style="line-height: 125%; margin: 0;"><span style="color: #c65d09; font-weight: bold;">root@637c83896b9d:/#</span> sudo apt-get install ssh
<span style="color: #c65d09; font-weight: bold;">root@637c83896b9d:/#</span> sudo apt-get install rsync
<span style="color: #c65d09; font-weight: bold;">root@637c83896b9d:/#</span> useradd hadoop -m -s /bin/bash
<span style="color: #c65d09; font-weight: bold;">root@637c83896b9d:/#</span> passwd hadoop
<span style="color: #888888;">Enter new UNIX password: </span>
<span style="color: #888888;">Retype new UNIX password: </span>
<span style="color: #888888;">passwd: password updated successfully</span>
<span style="color: #c65d09; font-weight: bold;">hadoop@637c83896b9d:/$</span> update-alternatives --config java
<span style="color: #888888;">There is only one alternative in link group java (providing /usr/bin/java): /usr/lib/jvm/java-8-oracle/jre/bin/java</span>
<span style="color: #888888;">Nothing to configure.</span>
<span style="color: #c65d09; font-weight: bold;">hadoop@637c83896b9d:/$</span> <span style="color: #996633;">JAVA_HOME</span><span style="color: #333333;">=</span>/usr/lib/jvm/java-8-oracle
<span style="color: #c65d09; font-weight: bold;">hadoop@637c83896b9d:/$</span> <span style="color: #007020;">export </span>JAVA_HOME
<span style="color: #c65d09; font-weight: bold;">hadoop@637c83896b9d:~$</span> tail .bashrc
<span style="color: #888888;">if ! shopt -oq posix; then</span>
<span style="color: #888888;"> if [ -f /usr/share/bash-completion/bash_completion ]; then</span>
<span style="color: #888888;"> . /usr/share/bash-completion/bash_completion</span>
<span style="color: #888888;"> elif [ -f /etc/bash_completion ]; then</span>
<span style="color: #888888;"> . /etc/bash_completion</span>
<span style="color: #888888;"> fi</span>
<span style="color: #888888;">fi</span>
<span style="color: #888888;">JAVA_HOME=/usr/lib/jvm/java-8-oracle</span>
<span style="color: #888888;">export JAVA_HOME</span>
<span style="color: #c65d09; font-weight: bold;">hadoop@637c83896b9d:~$</span>
</pre>
</div>
<br />
<!-- HTML generated using hilite.me --><br />
<div style="background: #ffffff; border-width: 0.1em 0.1em 0.1em 0.8em; border: solid gray; overflow: auto; padding: 0.2em 0.6em; width: auto;">
<pre style="line-height: 125%; margin: 0;"><span style="color: #c65d09; font-weight: bold;">hadoop@637c83896b9d:~$</span> <span style="color: #007020;">cd </span>hadoop-2.7.3/
<span style="color: #c65d09; font-weight: bold;">hadoop@637c83896b9d:~/hadoop-2.7.3$</span> <span style="color: #007020;">cd </span>bin
<span style="color: #c65d09; font-weight: bold;">hadoop@637c83896b9d:~/hadoop-2.7.3/bin$</span>
<span style="color: #c65d09; font-weight: bold;">hadoop@637c83896b9d:~/hadoop-2.7.3/bin$</span>
<span style="color: #c65d09; font-weight: bold;">hadoop@637c83896b9d:~/hadoop-2.7.3/bin$</span>
<span style="color: #c65d09; font-weight: bold;">hadoop@637c83896b9d:~/hadoop-2.7.3/bin$</span>
<span style="color: #c65d09; font-weight: bold;">hadoop@637c83896b9d:~/hadoop-2.7.3/bin$</span>
<span style="color: #c65d09; font-weight: bold;">hadoop@637c83896b9d:~/hadoop-2.7.3/bin$</span>
<span style="color: #c65d09; font-weight: bold;">hadoop@637c83896b9d:~/hadoop-2.7.3/bin$</span> ./hadoop
<span style="color: #888888;">Usage: hadoop [--config confdir] [COMMAND | CLASSNAME]</span>
<span style="color: #888888;"> CLASSNAME run the class named CLASSNAME</span>
<span style="color: #888888;"> or</span>
<span style="color: #888888;"> where COMMAND is one of:</span>
<span style="color: #888888;"> fs run a generic filesystem user client</span>
<span style="color: #888888;"> version print the version</span>
<span style="color: #888888;"> jar <jar> run a jar file</span>
<span style="color: #888888;"> note: please use "yarn jar" to launch</span>
<span style="color: #888888;"> YARN applications, not this command.</span>
<span style="color: #888888;"> checknative [-a|-h] check native hadoop and compression libraries availability</span>
<span style="color: #888888;"> distcp <srcurl> <desturl> copy file or directories recursively</span>
<span style="color: #888888;"> archive -archiveName NAME -p <parent path> <src>* <dest> create a hadoop archive</span>
<span style="color: #888888;"> classpath prints the class path needed to get the</span>
<span style="color: #888888;"> credential interact with credential providers</span>
<span style="color: #888888;"> Hadoop jar and the required libraries</span>
<span style="color: #888888;"> daemonlog get/set the log level for each daemon</span>
<span style="color: #888888;"> trace view and modify Hadoop tracing settings</span>
<span style="color: #888888;">Most commands print help when invoked w/o parameters.</span>
</pre>
</div>
<br />
<br />
<!-- HTML generated using hilite.me --><br />
<div style="background: #ffffff; border-width: 0.1em 0.1em 0.1em 0.8em; border: solid gray; overflow: auto; padding: 0.2em 0.6em; width: auto;">
<pre style="line-height: 125%; margin: 0;"><span style="color: #c65d09; font-weight: bold;">hadoop@637c83896b9d:~/hadoop-2.7.3$</span> <span style="color: #007020;">pwd</span>
<span style="color: #888888;">/home/hadoop/hadoop-2.7.3</span>
<span style="color: #c65d09; font-weight: bold;">hadoop@637c83896b9d:~/hadoop-2.7.3$</span> ls
<span style="color: #888888;">LICENSE.txt NOTICE.txt README.txt bin etc include lib libexec sbin share</span>
<span style="color: #c65d09; font-weight: bold;">hadoop@637c83896b9d:~/hadoop-2.7.3$</span>
<span style="color: #c65d09; font-weight: bold;">hadoop@637c83896b9d:~/hadoop-2.7.3$</span> mkdir input
<span style="color: #c65d09; font-weight: bold;">hadoop@637c83896b9d:~/hadoop-2.7.3$</span>
<span style="color: #c65d09; font-weight: bold;">hadoop@637c83896b9d:~/hadoop-2.7.3$</span> cp etc/hadoop/*.xml input
<span style="color: #c65d09; font-weight: bold;">hadoop@637c83896b9d:~/hadoop-2.7.3$</span>
<span style="color: #c65d09; font-weight: bold;">hadoop@637c83896b9d:~/hadoop-2.7.3$</span> bin/hadoop jar share/hadoop/mapreduce/hadoop-mapreduce-examples-2.7.3.jar grep input output <span style="background-color: #fff0f0;">'dfs[a-z.]+'</span>
<span style="color: #888888;">16/08/28 10:30:37 INFO Configuration.deprecation: session.id is deprecated. Instead, use dfs.metrics.session-id</span>
<span style="color: #888888;">16/08/28 10:30:37 INFO jvm.JvmMetrics: Initializing JVM Metrics with processName=JobTracker, sessionId=</span>
<span style="color: #888888;">16/08/28 10:30:37 INFO input.FileInputFormat: Total input paths to process : 8</span>
<span style="color: #888888;">16/08/28 10:30:37 INFO mapreduce.JobSubmitter: number of splits:8</span>
<span style="color: #888888;">16/08/28 10:30:38 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_local1669053990_0001</span>
<span style="color: #888888;">16/08/28 10:30:38 INFO mapreduce.Job: The url to track the job: http://localhost:8080/</span>
<span style="color: #888888;">16/08/28 10:30:38 INFO mapreduce.Job: Running job: job_local1669053990_0001</span>
<span style="color: #888888;">16/08/28 10:30:38 INFO mapred.LocalJobRunner: OutputCommitter set in config null</span>
<span style="color: #888888;">16/08/28 10:30:38 INFO output.FileOutputCommitter: File Output Committer Algorithm version is 1</span>
<span style="color: #888888;">16/08/28 10:30:38 INFO mapred.LocalJobRunner: OutputCommitter is org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter</span>
<span style="color: #888888;">16/08/28 10:30:38 INFO mapred.LocalJobRunner: Waiting for map tasks</span>
<span style="color: #888888;">16/08/28 10:30:38 INFO mapred.LocalJobRunner: Starting task: attempt_local1669053990_0001_m_000000_0</span>
<span style="color: #888888;">16/08/28 10:30:38 INFO output.FileOutputCommitter: File Output Committer Algorithm version is 1</span>
<span style="color: #888888;">16/08/28 10:30:38 INFO mapred.Task: Using ResourceCalculatorProcessTree : [ ]</span>
<span style="color: #888888;">16/08/28 10:30:38 INFO mapred.MapTask: Processing split: file:/home/hadoop/hadoop-2.7.3/input/hadoop-policy.xml:0+9683</span>
<span style="color: #888888;">16/08/28 10:30:38 INFO mapred.MapTask: (EQUATOR) 0 kvi 26214396(104857584)</span>
<span style="color: #888888;">16/08/28 10:30:38 INFO mapred.MapTask: mapreduce.task.io.sort.mb: 100</span>
<span style="color: #888888;">16/08/28 10:30:38 INFO mapred.MapTask: soft limit at 83886080</span>
<span style="color: #888888;">16/08/28 10:30:38 INFO mapred.MapTask: bufstart = 0; bufvoid = 104857600</span>
<span style="color: #888888;">16/08/28 10:30:38 INFO mapred.MapTask: kvstart = 26214396; length = 6553600</span>
<span style="color: #888888;">16/08/28 10:30:38 INFO mapred.MapTask: Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer</span>
<span style="color: #888888;">16/08/28 10:30:38 INFO mapred.LocalJobRunner: </span>
<span style="color: #888888;">16/08/28 10:30:38 INFO mapred.MapTask: Starting flush of map output</span>
<span style="color: #888888;">16/08/28 10:30:38 INFO mapred.MapTask: Spilling map output</span>
<span style="color: #888888;">16/08/28 10:30:38 INFO mapred.MapTask: bufstart = 0; bufend = 17; bufvoid = 104857600</span>
<span style="color: #888888;">16/08/28 10:30:38 INFO mapred.MapTask: kvstart = 26214396(104857584); kvend = 26214396(104857584); length = 1/6553600</span>
<span style="color: #888888;">16/08/28 10:30:38 INFO mapred.MapTask: Finished spill 0</span>
<span style="color: #888888;">16/08/28 10:30:38 INFO mapred.Task: Task:attempt_local1669053990_0001_m_000000_0 is done. And is in the process of committing</span>
<span style="color: #888888;">16/08/28 10:30:38 INFO mapred.LocalJobRunner: map</span>
<span style="color: #888888;">16/08/28 10:30:38 INFO mapred.Task: Task 'attempt_local1669053990_0001_m_000000_0' done.</span>
<span style="color: #888888;">16/08/28 10:30:38 INFO mapred.LocalJobRunner: Finishing task: attempt_local1669053990_0001_m_000000_0</span>
<span style="color: #888888;">16/08/28 10:30:38 INFO mapred.LocalJobRunner: Starting task: attempt_local1669053990_0001_m_000001_0</span>
<span style="color: #888888;">16/08/28 10:30:38 INFO output.FileOutputCommitter: File Output Committer Algorithm version is 1</span>
<span style="color: #888888;">16/08/28 10:30:38 INFO mapred.Task: Using ResourceCalculatorProcessTree : [ ]</span>
<span style="color: #888888;">16/08/28 10:30:38 INFO mapred.MapTask: Processing split: file:/home/hadoop/hadoop-2.7.3/input/kms-site.xml:0+5511</span>
<span style="color: #888888;">16/08/28 10:30:38 INFO mapred.MapTask: (EQUATOR) 0 kvi 26214396(104857584)</span>
<span style="color: #888888;">16/08/28 10:30:38 INFO mapred.MapTask: mapreduce.task.io.sort.mb: 100</span>
<span style="color: #888888;">16/08/28 10:30:38 INFO mapred.MapTask: soft limit at 83886080</span>
<span style="color: #888888;">16/08/28 10:30:38 INFO mapred.MapTask: bufstart = 0; bufvoid = 104857600</span>
<span style="color: #888888;">16/08/28 10:30:38 INFO mapred.MapTask: kvstart = 26214396; length = 6553600</span>
<span style="color: #888888;">16/08/28 10:30:38 INFO mapred.MapTask: Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer</span>
<span style="color: #888888;">16/08/28 10:30:38 INFO mapred.LocalJobRunner: </span>
<span style="color: #888888;">16/08/28 10:30:38 INFO mapred.MapTask: Starting flush of map output</span>
<span style="color: #888888;">16/08/28 10:30:38 INFO mapred.Task: Task:attempt_local1669053990_0001_m_000001_0 is done. And is in the process of committing</span>
<span style="color: #888888;">16/08/28 10:30:38 INFO mapred.LocalJobRunner: map</span>
<span style="color: #888888;">16/08/28 10:30:38 INFO mapred.Task: Task 'attempt_local1669053990_0001_m_000001_0' done.</span>
<span style="color: #888888;">16/08/28 10:30:38 INFO mapred.LocalJobRunner: Finishing task: attempt_local1669053990_0001_m_000001_0</span>
<span style="color: #888888;">16/08/28 10:30:38 INFO mapred.LocalJobRunner: Starting task: attempt_local1669053990_0001_m_000002_0</span>
<span style="color: #888888;">16/08/28 10:30:38 INFO output.FileOutputCommitter: File Output Committer Algorithm version is 1</span>
<span style="color: #888888;">16/08/28 10:30:38 INFO mapred.Task: Using ResourceCalculatorProcessTree : [ ]</span>
<span style="color: #888888;">16/08/28 10:30:38 INFO mapred.MapTask: Processing split: file:/home/hadoop/hadoop-2.7.3/input/capacity-scheduler.xml:0+4436</span>
<span style="color: #888888;">16/08/28 10:30:38 INFO mapred.MapTask: (EQUATOR) 0 kvi 26214396(104857584)</span>
<span style="color: #888888;">16/08/28 10:30:38 INFO mapred.MapTask: mapreduce.task.io.sort.mb: 100</span>
<span style="color: #888888;">16/08/28 10:30:38 INFO mapred.MapTask: soft limit at 83886080</span>
<span style="color: #888888;">16/08/28 10:30:38 INFO mapred.MapTask: bufstart = 0; bufvoid = 104857600</span>
<span style="color: #888888;">16/08/28 10:30:38 INFO mapred.MapTask: kvstart = 26214396; length = 6553600</span>
<span style="color: #888888;">16/08/28 10:30:38 INFO mapred.MapTask: Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer</span>
<span style="color: #888888;">16/08/28 10:30:38 INFO mapred.LocalJobRunner: </span>
<span style="color: #888888;">16/08/28 10:30:38 INFO mapred.MapTask: Starting flush of map output</span>
<span style="color: #888888;">16/08/28 10:30:38 INFO mapred.Task: Task:attempt_local1669053990_0001_m_000002_0 is done. And is in the process of committing</span>
<span style="color: #888888;">16/08/28 10:30:38 INFO mapred.LocalJobRunner: map</span>
<span style="color: #888888;">16/08/28 10:30:38 INFO mapred.Task: Task 'attempt_local1669053990_0001_m_000002_0' done.</span>
<span style="color: #888888;">16/08/28 10:30:38 INFO mapred.LocalJobRunner: Finishing task: attempt_local1669053990_0001_m_000002_0</span>
<span style="color: #888888;">16/08/28 10:30:38 INFO mapred.LocalJobRunner: Starting task: attempt_local1669053990_0001_m_000003_0</span>
<span style="color: #888888;">16/08/28 10:30:38 INFO output.FileOutputCommitter: File Output Committer Algorithm version is 1</span>
<span style="color: #888888;">16/08/28 10:30:38 INFO mapred.Task: Using ResourceCalculatorProcessTree : [ ]</span>
<span style="color: #888888;">16/08/28 10:30:38 INFO mapred.MapTask: Processing split: file:/home/hadoop/hadoop-2.7.3/input/kms-acls.xml:0+3518</span>
<span style="color: #888888;">16/08/28 10:30:38 INFO mapred.MapTask: (EQUATOR) 0 kvi 26214396(104857584)</span>
<span style="color: #888888;">16/08/28 10:30:38 INFO mapred.MapTask: mapreduce.task.io.sort.mb: 100</span>
<span style="color: #888888;">16/08/28 10:30:38 INFO mapred.MapTask: soft limit at 83886080</span>
<span style="color: #888888;">16/08/28 10:30:38 INFO mapred.MapTask: bufstart = 0; bufvoid = 104857600</span>
<span style="color: #888888;">16/08/28 10:30:38 INFO mapred.MapTask: kvstart = 26214396; length = 6553600</span>
<span style="color: #888888;">16/08/28 10:30:38 INFO mapred.MapTask: Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer</span>
<span style="color: #888888;">16/08/28 10:30:38 INFO mapred.LocalJobRunner: </span>
<span style="color: #888888;">16/08/28 10:30:38 INFO mapred.MapTask: Starting flush of map output</span>
<span style="color: #888888;">16/08/28 10:30:38 INFO mapred.Task: Task:attempt_local1669053990_0001_m_000003_0 is done. And is in the process of committing</span>
<span style="color: #888888;">16/08/28 10:30:38 INFO mapred.LocalJobRunner: map</span>
<span style="color: #888888;">16/08/28 10:30:38 INFO mapred.Task: Task 'attempt_local1669053990_0001_m_000003_0' done.</span>
<span style="color: #888888;">16/08/28 10:30:38 INFO mapred.LocalJobRunner: Finishing task: attempt_local1669053990_0001_m_000003_0</span>
<span style="color: #888888;">16/08/28 10:30:38 INFO mapred.LocalJobRunner: Starting task: attempt_local1669053990_0001_m_000004_0</span>
<span style="color: #888888;">16/08/28 10:30:38 INFO output.FileOutputCommitter: File Output Committer Algorithm version is 1</span>
<span style="color: #888888;">16/08/28 10:30:38 INFO mapred.Task: Using ResourceCalculatorProcessTree : [ ]</span>
<span style="color: #888888;">16/08/28 10:30:38 INFO mapred.MapTask: Processing split: file:/home/hadoop/hadoop-2.7.3/input/hdfs-site.xml:0+775</span>
<span style="color: #888888;">16/08/28 10:30:39 INFO mapred.MapTask: (EQUATOR) 0 kvi 26214396(104857584)</span>
<span style="color: #888888;">16/08/28 10:30:39 INFO mapred.MapTask: mapreduce.task.io.sort.mb: 100</span>
<span style="color: #888888;">16/08/28 10:30:39 INFO mapred.MapTask: soft limit at 83886080</span>
<span style="color: #888888;">16/08/28 10:30:39 INFO mapred.MapTask: bufstart = 0; bufvoid = 104857600</span>
<span style="color: #888888;">16/08/28 10:30:39 INFO mapred.MapTask: kvstart = 26214396; length = 6553600</span>
<span style="color: #888888;">16/08/28 10:30:39 INFO mapred.MapTask: Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer</span>
<span style="color: #888888;">16/08/28 10:30:39 INFO mapred.LocalJobRunner: </span>
<span style="color: #888888;">16/08/28 10:30:39 INFO mapred.MapTask: Starting flush of map output</span>
<span style="color: #888888;">16/08/28 10:30:39 INFO mapred.Task: Task:attempt_local1669053990_0001_m_000004_0 is done. And is in the process of committing</span>
<span style="color: #888888;">16/08/28 10:30:39 INFO mapred.LocalJobRunner: map</span>
<span style="color: #888888;">16/08/28 10:30:39 INFO mapred.Task: Task 'attempt_local1669053990_0001_m_000004_0' done.</span>
<span style="color: #888888;">16/08/28 10:30:39 INFO mapred.LocalJobRunner: Finishing task: attempt_local1669053990_0001_m_000004_0</span>
<span style="color: #888888;">16/08/28 10:30:39 INFO mapred.LocalJobRunner: Starting task: attempt_local1669053990_0001_m_000005_0</span>
<span style="color: #888888;">16/08/28 10:30:39 INFO output.FileOutputCommitter: File Output Committer Algorithm version is 1</span>
<span style="color: #888888;">16/08/28 10:30:39 INFO mapred.Task: Using ResourceCalculatorProcessTree : [ ]</span>
<span style="color: #888888;">16/08/28 10:30:39 INFO mapred.MapTask: Processing split: file:/home/hadoop/hadoop-2.7.3/input/core-site.xml:0+774</span>
<span style="color: #888888;">16/08/28 10:30:39 INFO mapred.MapTask: (EQUATOR) 0 kvi 26214396(104857584)</span>
<span style="color: #888888;">16/08/28 10:30:39 INFO mapred.MapTask: mapreduce.task.io.sort.mb: 100</span>
<span style="color: #888888;">16/08/28 10:30:39 INFO mapred.MapTask: soft limit at 83886080</span>
<span style="color: #888888;">16/08/28 10:30:39 INFO mapred.MapTask: bufstart = 0; bufvoid = 104857600</span>
<span style="color: #888888;">16/08/28 10:30:39 INFO mapred.MapTask: kvstart = 26214396; length = 6553600</span>
<span style="color: #888888;">16/08/28 10:30:39 INFO mapred.MapTask: Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer</span>
<span style="color: #888888;">16/08/28 10:30:39 INFO mapred.LocalJobRunner: </span>
<span style="color: #888888;">16/08/28 10:30:39 INFO mapred.MapTask: Starting flush of map output</span>
<span style="color: #888888;">16/08/28 10:30:39 INFO mapred.Task: Task:attempt_local1669053990_0001_m_000005_0 is done. And is in the process of committing</span>
<span style="color: #888888;">16/08/28 10:30:39 INFO mapred.LocalJobRunner: map</span>
<span style="color: #888888;">16/08/28 10:30:39 INFO mapred.Task: Task 'attempt_local1669053990_0001_m_000005_0' done.</span>
<span style="color: #888888;">16/08/28 10:30:39 INFO mapred.LocalJobRunner: Finishing task: attempt_local1669053990_0001_m_000005_0</span>
<span style="color: #888888;">16/08/28 10:30:39 INFO mapred.LocalJobRunner: Starting task: attempt_local1669053990_0001_m_000006_0</span>
<span style="color: #888888;">16/08/28 10:30:39 INFO output.FileOutputCommitter: File Output Committer Algorithm version is 1</span>
<span style="color: #888888;">16/08/28 10:30:39 INFO mapred.Task: Using ResourceCalculatorProcessTree : [ ]</span>
<span style="color: #888888;">16/08/28 10:30:39 INFO mapred.MapTask: Processing split: file:/home/hadoop/hadoop-2.7.3/input/yarn-site.xml:0+690</span>
<span style="color: #888888;">16/08/28 10:30:39 INFO mapred.MapTask: (EQUATOR) 0 kvi 26214396(104857584)</span>
<span style="color: #888888;">16/08/28 10:30:39 INFO mapred.MapTask: mapreduce.task.io.sort.mb: 100</span>
<span style="color: #888888;">16/08/28 10:30:39 INFO mapred.MapTask: soft limit at 83886080</span>
<span style="color: #888888;">16/08/28 10:30:39 INFO mapred.MapTask: bufstart = 0; bufvoid = 104857600</span>
<span style="color: #888888;">16/08/28 10:30:39 INFO mapred.MapTask: kvstart = 26214396; length = 6553600</span>
<span style="color: #888888;">16/08/28 10:30:39 INFO mapred.MapTask: Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer</span>
<span style="color: #888888;">16/08/28 10:30:39 INFO mapred.LocalJobRunner: </span>
<span style="color: #888888;">16/08/28 10:30:39 INFO mapred.MapTask: Starting flush of map output</span>
<span style="color: #888888;">16/08/28 10:30:39 INFO mapred.Task: Task:attempt_local1669053990_0001_m_000006_0 is done. And is in the process of committing</span>
<span style="color: #888888;">16/08/28 10:30:39 INFO mapred.LocalJobRunner: map</span>
<span style="color: #888888;">16/08/28 10:30:39 INFO mapred.Task: Task 'attempt_local1669053990_0001_m_000006_0' done.</span>
<span style="color: #888888;">16/08/28 10:30:39 INFO mapred.LocalJobRunner: Finishing task: attempt_local1669053990_0001_m_000006_0</span>
<span style="color: #888888;">16/08/28 10:30:39 INFO mapred.LocalJobRunner: Starting task: attempt_local1669053990_0001_m_000007_0</span>
<span style="color: #888888;">16/08/28 10:30:39 INFO output.FileOutputCommitter: File Output Committer Algorithm version is 1</span>
<span style="color: #888888;">16/08/28 10:30:39 INFO mapred.Task: Using ResourceCalculatorProcessTree : [ ]</span>
<span style="color: #888888;">16/08/28 10:30:39 INFO mapred.MapTask: Processing split: file:/home/hadoop/hadoop-2.7.3/input/httpfs-site.xml:0+620</span>
<span style="color: #888888;">16/08/28 10:30:39 INFO mapred.MapTask: (EQUATOR) 0 kvi 26214396(104857584)</span>
<span style="color: #888888;">16/08/28 10:30:39 INFO mapred.MapTask: mapreduce.task.io.sort.mb: 100</span>
<span style="color: #888888;">16/08/28 10:30:39 INFO mapred.MapTask: soft limit at 83886080</span>
<span style="color: #888888;">16/08/28 10:30:39 INFO mapred.MapTask: bufstart = 0; bufvoid = 104857600</span>
<span style="color: #888888;">16/08/28 10:30:39 INFO mapred.MapTask: kvstart = 26214396; length = 6553600</span>
<span style="color: #888888;">16/08/28 10:30:39 INFO mapred.MapTask: Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer</span>
<span style="color: #888888;">16/08/28 10:30:39 INFO mapred.LocalJobRunner: </span>
<span style="color: #888888;">16/08/28 10:30:39 INFO mapred.MapTask: Starting flush of map output</span>
<span style="color: #888888;">16/08/28 10:30:39 INFO mapred.Task: Task:attempt_local1669053990_0001_m_000007_0 is done. And is in the process of committing</span>
<span style="color: #888888;">16/08/28 10:30:39 INFO mapred.LocalJobRunner: map</span>
<span style="color: #888888;">16/08/28 10:30:39 INFO mapred.Task: Task 'attempt_local1669053990_0001_m_000007_0' done.</span>
<span style="color: #888888;">16/08/28 10:30:39 INFO mapred.LocalJobRunner: Finishing task: attempt_local1669053990_0001_m_000007_0</span>
<span style="color: #888888;">16/08/28 10:30:39 INFO mapred.LocalJobRunner: map task executor complete.</span>
<span style="color: #888888;">16/08/28 10:30:39 INFO mapred.LocalJobRunner: Waiting for reduce tasks</span>
<span style="color: #888888;">16/08/28 10:30:39 INFO mapred.LocalJobRunner: Starting task: attempt_local1669053990_0001_r_000000_0</span>
<span style="color: #888888;">16/08/28 10:30:39 INFO output.FileOutputCommitter: File Output Committer Algorithm version is 1</span>
<span style="color: #888888;">16/08/28 10:30:39 INFO mapred.Task: Using ResourceCalculatorProcessTree : [ ]</span>
<span style="color: #888888;">16/08/28 10:30:39 INFO mapred.ReduceTask: Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@4acf1a45</span>
<span style="color: #888888;">16/08/28 10:30:39 INFO reduce.MergeManagerImpl: MergerManager: memoryLimit=334338464, maxSingleShuffleLimit=83584616, mergeThreshold=220663392, ioSortFactor=10, memToMemMergeOutputsThreshold=10</span>
<span style="color: #888888;">16/08/28 10:30:39 INFO reduce.EventFetcher: attempt_local1669053990_0001_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events</span>
<span style="color: #888888;">16/08/28 10:30:39 INFO reduce.LocalFetcher: localfetcher#1 about to shuffle output of map attempt_local1669053990_0001_m_000007_0 decomp: 2 len: 6 to MEMORY</span>
<span style="color: #888888;">16/08/28 10:30:39 INFO reduce.InMemoryMapOutput: Read 2 bytes from map-output for attempt_local1669053990_0001_m_000007_0</span>
<span style="color: orange;"><span style="color: #888888;">16/08/28 10:30:39 WARN io.ReadaheadPool: Failed readahead on ifile</span>
<span style="color: #888888;">EBADF: Bad file descriptor</span>
<span style="color: #888888;"> at org.apache.hadoop.io.nativeio.NativeIO$POSIX.posix_fadvise(Native Method)</span>
<span style="color: #888888;"> at org.apache.hadoop.io.nativeio.NativeIO$POSIX.posixFadviseIfPossible(NativeIO.java:267)</span>
<span style="color: #888888;"> at org.apache.hadoop.io.nativeio.NativeIO$POSIX$CacheManipulator.posixFadviseIfPossible(NativeIO.java:146)</span>
<span style="color: #888888;"> at org.apache.hadoop.io.ReadaheadPool$ReadaheadRequestImpl.run(ReadaheadPool.java:206)</span>
<span style="color: #888888;"> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)</span>
<span style="color: #888888;"> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)</span>
<span style="color: #888888;"> at java.lang.Thread.run(Thread.java:745)</span></span>
<span style="color: #888888;">16/08/28 10:30:39 INFO reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 2, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->2</span>
<span style="color: #888888;">16/08/28 10:30:39 INFO reduce.LocalFetcher: localfetcher#1 about to shuffle output of map attempt_local1669053990_0001_m_000004_0 decomp: 2 len: 6 to MEMORY</span>
<span style="color: #888888;">16/08/28 10:30:39 INFO reduce.InMemoryMapOutput: Read 2 bytes from map-output for attempt_local1669053990_0001_m_000004_0</span>
<span style="color: #888888;">16/08/28 10:30:39 INFO reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 2, inMemoryMapOutputs.size() -> 2, commitMemory -> 2, usedMemory ->4</span>
<span style="color: #888888;">16/08/28 10:30:39 INFO reduce.LocalFetcher: localfetcher#1 about to shuffle output of map attempt_local1669053990_0001_m_000001_0 decomp: 2 len: 6 to MEMORY</span>
<span style="color: #888888;">16/08/28 10:30:39 INFO reduce.InMemoryMapOutput: Read 2 bytes from map-output for attempt_local1669053990_0001_m_000001_0</span>
<span style="color: #888888;">16/08/28 10:30:39 INFO reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 2, inMemoryMapOutputs.size() -> 3, commitMemory -> 4, usedMemory ->6</span>
<span style="color: #888888;">16/08/28 10:30:39 INFO reduce.LocalFetcher: localfetcher#1 about to shuffle output of map attempt_local1669053990_0001_m_000005_0 decomp: 2 len: 6 to MEMORY</span>
<span style="color: #888888;">16/08/28 10:30:39 INFO reduce.InMemoryMapOutput: Read 2 bytes from map-output for attempt_local1669053990_0001_m_000005_0</span>
<span style="color: #888888;">16/08/28 10:30:39 INFO reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 2, inMemoryMapOutputs.size() -> 4, commitMemory -> 6, usedMemory ->8</span>
<span style="color: #888888;">16/08/28 10:30:39 INFO reduce.LocalFetcher: localfetcher#1 about to shuffle output of map attempt_local1669053990_0001_m_000002_0 decomp: 2 len: 6 to MEMORY</span>
<span style="color: #888888;">16/08/28 10:30:39 INFO reduce.InMemoryMapOutput: Read 2 bytes from map-output for attempt_local1669053990_0001_m_000002_0</span>
<span style="color: #888888;">16/08/28 10:30:39 INFO reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 2, inMemoryMapOutputs.size() -> 5, commitMemory -> 8, usedMemory ->10</span>
<span style="color: orange;"><span style="color: #888888;">16/08/28 10:30:39 WARN io.ReadaheadPool: Failed readahead on ifile</span>
<span style="color: #888888;">EBADF: Bad file descriptor</span>
<span style="color: #888888;"> at org.apache.hadoop.io.nativeio.NativeIO$POSIX.posix_fadvise(Native Method)</span>
<span style="color: #888888;"> at org.apache.hadoop.io.nativeio.NativeIO$POSIX.posixFadviseIfPossible(NativeIO.java:267)</span>
<span style="color: #888888;"> at org.apache.hadoop.io.nativeio.NativeIO$POSIX$CacheManipulator.posixFadviseIfPossible(NativeIO.java:146)</span>
<span style="color: #888888;"> at org.apache.hadoop.io.ReadaheadPool$ReadaheadRequestImpl.run(ReadaheadPool.java:206)</span>
<span style="color: #888888;"> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)</span>
<span style="color: #888888;"> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)</span>
<span style="color: #888888;"> at java.lang.Thread.run(Thread.java:745)</span></span>
<span style="color: #888888;">16/08/28 10:30:39 INFO reduce.LocalFetcher: localfetcher#1 about to shuffle output of map attempt_local1669053990_0001_m_000006_0 decomp: 2 len: 6 to MEMORY</span>
<span style="color: #888888;">16/08/28 10:30:39 INFO reduce.InMemoryMapOutput: Read 2 bytes from map-output for attempt_local1669053990_0001_m_000006_0</span>
<span style="color: #888888;">16/08/28 10:30:39 INFO reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 2, inMemoryMapOutputs.size() -> 6, commitMemory -> 10, usedMemory ->12</span>
<span style="color: orange;"><span style="color: #888888;">16/08/28 10:30:39 WARN io.ReadaheadPool: Failed readahead on ifile</span>
<span style="color: #888888;">EBADF: Bad file descriptor</span>
<span style="color: #888888;"> at org.apache.hadoop.io.nativeio.NativeIO$POSIX.posix_fadvise(Native Method)</span>
<span style="color: #888888;"> at org.apache.hadoop.io.nativeio.NativeIO$POSIX.posixFadviseIfPossible(NativeIO.java:267)</span>
<span style="color: #888888;"> at org.apache.hadoop.io.nativeio.NativeIO$POSIX$CacheManipulator.posixFadviseIfPossible(NativeIO.java:146)</span>
<span style="color: #888888;"> at org.apache.hadoop.io.ReadaheadPool$ReadaheadRequestImpl.run(ReadaheadPool.java:206)</span>
<span style="color: #888888;"> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)</span>
<span style="color: #888888;"> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)</span>
<span style="color: #888888;"> at java.lang.Thread.run(Thread.java:745)</span></span>
<span style="color: #888888;">16/08/28 10:30:39 INFO reduce.LocalFetcher: localfetcher#1 about to shuffle output of map attempt_local1669053990_0001_m_000003_0 decomp: 2 len: 6 to MEMORY</span>
<span style="color: #888888;">16/08/28 10:30:39 INFO reduce.InMemoryMapOutput: Read 2 bytes from map-output for attempt_local1669053990_0001_m_000003_0</span>
<span style="color: #888888;">16/08/28 10:30:39 INFO reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 2, inMemoryMapOutputs.size() -> 7, commitMemory -> 12, usedMemory ->14</span>
<span style="color: #888888;">16/08/28 10:30:39 INFO reduce.LocalFetcher: localfetcher#1 about to shuffle output of map attempt_local1669053990_0001_m_000000_0 decomp: 21 len: 25 to MEMORY</span>
<span style="color: #888888;">16/08/28 10:30:39 INFO reduce.InMemoryMapOutput: Read 21 bytes from map-output for attempt_local1669053990_0001_m_000000_0</span>
<span style="color: #888888;">16/08/28 10:30:39 INFO reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 21, inMemoryMapOutputs.size() -> 8, commitMemory -> 14, usedMemory ->35</span>
<span style="color: #888888;">16/08/28 10:30:39 INFO reduce.EventFetcher: EventFetcher is interrupted.. Returning</span>
<span style="color: #888888;">16/08/28 10:30:39 INFO mapred.LocalJobRunner: 8 / 8 copied.</span>
<span style="color: #888888;">16/08/28 10:30:39 INFO reduce.MergeManagerImpl: finalMerge called with 8 in-memory map-outputs and 0 on-disk map-outputs</span>
<span style="color: #888888;">16/08/28 10:30:39 INFO mapred.Merger: Merging 8 sorted segments</span>
<span style="color: #888888;">16/08/28 10:30:39 INFO mapred.Merger: Down to the last merge-pass, with 1 segments left of total size: 10 bytes</span>
<span style="color: #888888;">16/08/28 10:30:39 INFO reduce.MergeManagerImpl: Merged 8 segments, 35 bytes to disk to satisfy reduce memory limit</span>
<span style="color: #888888;">16/08/28 10:30:39 INFO reduce.MergeManagerImpl: Merging 1 files, 25 bytes from disk</span>
<span style="color: #888888;">16/08/28 10:30:39 INFO reduce.MergeManagerImpl: Merging 0 segments, 0 bytes from memory into reduce</span>
<span style="color: #888888;">16/08/28 10:30:39 INFO mapred.Merger: Merging 1 sorted segments</span>
<span style="color: #888888;">16/08/28 10:30:39 INFO mapred.Merger: Down to the last merge-pass, with 1 segments left of total size: 10 bytes</span>
<span style="color: #888888;">16/08/28 10:30:39 INFO mapred.LocalJobRunner: 8 / 8 copied.</span>
<span style="color: #888888;">16/08/28 10:30:39 INFO Configuration.deprecation: mapred.skip.on is deprecated. Instead, use mapreduce.job.skiprecords</span>
<span style="color: #888888;">16/08/28 10:30:39 INFO mapred.Task: Task:attempt_local1669053990_0001_r_000000_0 is done. And is in the process of committing</span>
<span style="color: #888888;">16/08/28 10:30:39 INFO mapred.LocalJobRunner: 8 / 8 copied.</span>
<span style="color: #888888;">16/08/28 10:30:39 INFO mapred.Task: Task attempt_local1669053990_0001_r_000000_0 is allowed to commit now</span>
<span style="color: #888888;">16/08/28 10:30:39 INFO output.FileOutputCommitter: Saved output of task 'attempt_local1669053990_0001_r_000000_0' to file:/home/hadoop/hadoop-2.7.3/grep-temp-134950757/_temporary/0/task_local1669053990_0001_r_000000</span>
<span style="color: #888888;">16/08/28 10:30:39 INFO mapred.LocalJobRunner: reduce > reduce</span>
<span style="color: #888888;">16/08/28 10:30:39 INFO mapred.Task: Task 'attempt_local1669053990_0001_r_000000_0' done.</span>
<span style="color: #888888;">16/08/28 10:30:39 INFO mapred.LocalJobRunner: Finishing task: attempt_local1669053990_0001_r_000000_0</span>
<span style="color: #888888;">16/08/28 10:30:39 INFO mapred.LocalJobRunner: reduce task executor complete.</span>
<span style="color: #888888;">16/08/28 10:30:39 INFO mapreduce.Job: Job job_local1669053990_0001 running in uber mode : false</span>
<span style="color: #888888;">16/08/28 10:30:39 INFO mapreduce.Job: map 100% reduce 100%</span>
<span style="color: #888888;">16/08/28 10:30:39 INFO mapreduce.Job: Job job_local1669053990_0001 completed successfully</span>
<span style="color: #888888;">16/08/28 10:30:39 INFO mapreduce.Job: Counters: 30</span>
<span style="color: #888888;"> File System Counters</span>
<span style="color: #888888;"> FILE: Number of bytes read=2895165</span>
<span style="color: #888888;"> FILE: Number of bytes written=5259782</span>
<span style="color: #888888;"> FILE: Number of read operations=0</span>
<span style="color: #888888;"> FILE: Number of large read operations=0</span>
<span style="color: #888888;"> FILE: Number of write operations=0</span>
<span style="color: #888888;"> Map-Reduce Framework</span>
<span style="color: #888888;"> Map input records=745</span>
<span style="color: #888888;"> Map output records=1</span>
<span style="color: #888888;"> Map output bytes=17</span>
<span style="color: #888888;"> Map output materialized bytes=67</span>
<span style="color: #888888;"> Input split bytes=933</span>
<span style="color: #888888;"> Combine input records=1</span>
<span style="color: #888888;"> Combine output records=1</span>
<span style="color: #888888;"> Reduce input groups=1</span>
<span style="color: #888888;"> Reduce shuffle bytes=67</span>
<span style="color: #888888;"> Reduce input records=1</span>
<span style="color: #888888;"> Reduce output records=1</span>
<span style="color: #888888;"> Spilled Records=2</span>
<span style="color: #888888;"> Shuffled Maps =8</span>
<span style="color: #888888;"> Failed Shuffles=0</span>
<span style="color: #888888;"> Merged Map outputs=8</span>
<span style="color: #888888;"> GC time elapsed (ms)=85</span>
<span style="color: #888888;"> Total committed heap usage (bytes)=2771386368</span>
<span style="color: #888888;"> Shuffle Errors</span>
<span style="color: #888888;"> BAD_ID=0</span>
<span style="color: #888888;"> CONNECTION=0</span>
<span style="color: #888888;"> IO_ERROR=0</span>
<span style="color: #888888;"> WRONG_LENGTH=0</span>
<span style="color: #888888;"> WRONG_MAP=0</span>
<span style="color: #888888;"> WRONG_REDUCE=0</span>
<span style="color: #888888;"> File Input Format Counters </span>
<span style="color: #888888;"> Bytes Read=26007</span>
<span style="color: #888888;"> File Output Format Counters </span>
<span style="color: #888888;"> Bytes Written=123</span>
<span style="color: #888888;">16/08/28 10:30:39 INFO jvm.JvmMetrics: Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized</span>
<span style="color: #888888;">16/08/28 10:30:39 INFO input.FileInputFormat: Total input paths to process : 1</span>
<span style="color: #888888;">16/08/28 10:30:39 INFO mapreduce.JobSubmitter: number of splits:1</span>
<span style="color: #888888;">16/08/28 10:30:39 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_local2056692232_0002</span>
<span style="color: #888888;">16/08/28 10:30:39 INFO mapreduce.Job: The url to track the job: http://localhost:8080/</span>
<span style="color: #888888;">16/08/28 10:30:39 INFO mapreduce.Job: Running job: job_local2056692232_0002</span>
<span style="color: #888888;">16/08/28 10:30:39 INFO mapred.LocalJobRunner: OutputCommitter set in config null</span>
<span style="color: #888888;">16/08/28 10:30:39 INFO output.FileOutputCommitter: File Output Committer Algorithm version is 1</span>
<span style="color: #888888;">16/08/28 10:30:39 INFO mapred.LocalJobRunner: OutputCommitter is org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter</span>
<span style="color: #888888;">16/08/28 10:30:39 INFO mapred.LocalJobRunner: Waiting for map tasks</span>
<span style="color: #888888;">16/08/28 10:30:39 INFO mapred.LocalJobRunner: Starting task: attempt_local2056692232_0002_m_000000_0</span>
<span style="color: #888888;">16/08/28 10:30:39 INFO output.FileOutputCommitter: File Output Committer Algorithm version is 1</span>
<span style="color: #888888;">16/08/28 10:30:39 INFO mapred.Task: Using ResourceCalculatorProcessTree : [ ]</span>
<span style="color: #888888;">16/08/28 10:30:39 INFO mapred.MapTask: Processing split: file:/home/hadoop/hadoop-2.7.3/grep-temp-134950757/part-r-00000:0+111</span>
<span style="color: #888888;">16/08/28 10:30:39 INFO mapred.MapTask: (EQUATOR) 0 kvi 26214396(104857584)</span>
<span style="color: #888888;">16/08/28 10:30:39 INFO mapred.MapTask: mapreduce.task.io.sort.mb: 100</span>
<span style="color: #888888;">16/08/28 10:30:39 INFO mapred.MapTask: soft limit at 83886080</span>
<span style="color: #888888;">16/08/28 10:30:39 INFO mapred.MapTask: bufstart = 0; bufvoid = 104857600</span>
<span style="color: #888888;">16/08/28 10:30:39 INFO mapred.MapTask: kvstart = 26214396; length = 6553600</span>
<span style="color: #888888;">16/08/28 10:30:39 INFO mapred.MapTask: Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer</span>
<span style="color: #888888;">16/08/28 10:30:39 INFO mapred.LocalJobRunner: </span>
<span style="color: #888888;">16/08/28 10:30:39 INFO mapred.MapTask: Starting flush of map output</span>
<span style="color: #888888;">16/08/28 10:30:39 INFO mapred.MapTask: Spilling map output</span>
<span style="color: #888888;">16/08/28 10:30:39 INFO mapred.MapTask: bufstart = 0; bufend = 17; bufvoid = 104857600</span>
<span style="color: #888888;">16/08/28 10:30:39 INFO mapred.MapTask: kvstart = 26214396(104857584); kvend = 26214396(104857584); length = 1/6553600</span>
<span style="color: #888888;">16/08/28 10:30:39 INFO mapred.MapTask: Finished spill 0</span>
<span style="color: #888888;">16/08/28 10:30:39 INFO mapred.Task: Task:attempt_local2056692232_0002_m_000000_0 is done. And is in the process of committing</span>
<span style="color: #888888;">16/08/28 10:30:39 INFO mapred.LocalJobRunner: map</span>
<span style="color: #888888;">16/08/28 10:30:39 INFO mapred.Task: Task 'attempt_local2056692232_0002_m_000000_0' done.</span>
<span style="color: #888888;">16/08/28 10:30:39 INFO mapred.LocalJobRunner: Finishing task: attempt_local2056692232_0002_m_000000_0</span>
<span style="color: #888888;">16/08/28 10:30:39 INFO mapred.LocalJobRunner: map task executor complete.</span>
<span style="color: #888888;">16/08/28 10:30:39 INFO mapred.LocalJobRunner: Waiting for reduce tasks</span>
<span style="color: #888888;">16/08/28 10:30:39 INFO mapred.LocalJobRunner: Starting task: attempt_local2056692232_0002_r_000000_0</span>
<span style="color: #888888;">16/08/28 10:30:39 INFO output.FileOutputCommitter: File Output Committer Algorithm version is 1</span>
<span style="color: #888888;">16/08/28 10:30:39 INFO mapred.Task: Using ResourceCalculatorProcessTree : [ ]</span>
<span style="color: #888888;">16/08/28 10:30:39 INFO mapred.ReduceTask: Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@3f72041c</span>
<span style="color: #888888;">16/08/28 10:30:39 INFO reduce.MergeManagerImpl: MergerManager: memoryLimit=334338464, maxSingleShuffleLimit=83584616, mergeThreshold=220663392, ioSortFactor=10, memToMemMergeOutputsThreshold=10</span>
<span style="color: #888888;">16/08/28 10:30:39 INFO reduce.EventFetcher: attempt_local2056692232_0002_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events</span>
<span style="color: #888888;">16/08/28 10:30:39 INFO reduce.LocalFetcher: localfetcher#2 about to shuffle output of map attempt_local2056692232_0002_m_000000_0 decomp: 21 len: 25 to MEMORY</span>
<span style="color: #888888;">16/08/28 10:30:39 INFO reduce.InMemoryMapOutput: Read 21 bytes from map-output for attempt_local2056692232_0002_m_000000_0</span>
<span style="color: #888888;">16/08/28 10:30:39 INFO reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 21, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->21</span>
<span style="color: #888888;">16/08/28 10:30:39 INFO reduce.EventFetcher: EventFetcher is interrupted.. Returning</span>
<span style="color: #888888;">16/08/28 10:30:39 INFO mapred.LocalJobRunner: 1 / 1 copied.</span>
<span style="color: #888888;">16/08/28 10:30:39 INFO reduce.MergeManagerImpl: finalMerge called with 1 in-memory map-outputs and 0 on-disk map-outputs</span>
<span style="color: #888888;">16/08/28 10:30:39 INFO mapred.Merger: Merging 1 sorted segments</span>
<span style="color: #888888;">16/08/28 10:30:39 INFO mapred.Merger: Down to the last merge-pass, with 1 segments left of total size: 11 bytes</span>
<span style="color: #888888;">16/08/28 10:30:39 INFO reduce.MergeManagerImpl: Merged 1 segments, 21 bytes to disk to satisfy reduce memory limit</span>
<span style="color: #888888;">16/08/28 10:30:39 INFO reduce.MergeManagerImpl: Merging 1 files, 25 bytes from disk</span>
<span style="color: #888888;">16/08/28 10:30:39 INFO reduce.MergeManagerImpl: Merging 0 segments, 0 bytes from memory into reduce</span>
<span style="color: #888888;">16/08/28 10:30:39 INFO mapred.Merger: Merging 1 sorted segments</span>
<span style="color: #888888;">16/08/28 10:30:39 INFO mapred.Merger: Down to the last merge-pass, with 1 segments left of total size: 11 bytes</span>
<span style="color: #888888;">16/08/28 10:30:39 INFO mapred.LocalJobRunner: 1 / 1 copied.</span>
<span style="color: #888888;">16/08/28 10:30:39 INFO mapred.Task: Task:attempt_local2056692232_0002_r_000000_0 is done. And is in the process of committing</span>
<span style="color: #888888;">16/08/28 10:30:39 INFO mapred.LocalJobRunner: 1 / 1 copied.</span>
<span style="color: #888888;">16/08/28 10:30:39 INFO mapred.Task: Task attempt_local2056692232_0002_r_000000_0 is allowed to commit now</span>
<span style="color: #888888;">16/08/28 10:30:39 INFO output.FileOutputCommitter: Saved output of task 'attempt_local2056692232_0002_r_000000_0' to file:/home/hadoop/hadoop-2.7.3/output/_temporary/0/task_local2056692232_0002_r_000000</span>
<span style="color: #888888;">16/08/28 10:30:39 INFO mapred.LocalJobRunner: reduce > reduce</span>
<span style="color: #888888;">16/08/28 10:30:39 INFO mapred.Task: Task 'attempt_local2056692232_0002_r_000000_0' done.</span>
<span style="color: #888888;">16/08/28 10:30:39 INFO mapred.LocalJobRunner: Finishing task: attempt_local2056692232_0002_r_000000_0</span>
<span style="color: #888888;">16/08/28 10:30:39 INFO mapred.LocalJobRunner: reduce task executor complete.</span>
<span style="color: #888888;">16/08/28 10:30:40 INFO mapreduce.Job: Job job_local2056692232_0002 running in uber mode : false</span>
<span style="color: #888888;">16/08/28 10:30:40 INFO mapreduce.Job: map 100% reduce 100%</span>
<span style="color: #888888;">16/08/28 10:30:40 INFO mapreduce.Job: Job job_local2056692232_0002 completed successfully</span>
<span style="color: #888888;">16/08/28 10:30:40 INFO mapreduce.Job: Counters: 30</span>
<span style="color: #888888;"> File System Counters</span>
<span style="color: #888888;"> FILE: Number of bytes read=1249180</span>
<span style="color: #888888;"> FILE: Number of bytes written=2333622</span>
<span style="color: #888888;"> FILE: Number of read operations=0</span>
<span style="color: #888888;"> FILE: Number of large read operations=0</span>
<span style="color: #888888;"> FILE: Number of write operations=0</span>
<span style="color: #888888;"> Map-Reduce Framework</span>
<span style="color: #888888;"> Map input records=1</span>
<span style="color: #888888;"> Map output records=1</span>
<span style="color: #888888;"> Map output bytes=17</span>
<span style="color: #888888;"> Map output materialized bytes=25</span>
<span style="color: #888888;"> Input split bytes=128</span>
<span style="color: #888888;"> Combine input records=0</span>
<span style="color: #888888;"> Combine output records=0</span>
<span style="color: #888888;"> Reduce input groups=1</span>
<span style="color: #888888;"> Reduce shuffle bytes=25</span>
<span style="color: #888888;"> Reduce input records=1</span>
<span style="color: #888888;"> Reduce output records=1</span>
<span style="color: #888888;"> Spilled Records=2</span>
<span style="color: #888888;"> Shuffled Maps =1</span>
<span style="color: #888888;"> Failed Shuffles=0</span>
<span style="color: #888888;"> Merged Map outputs=1</span>
<span style="color: #888888;"> GC time elapsed (ms)=0</span>
<span style="color: #888888;"> Total committed heap usage (bytes)=896532480</span>
<span style="color: #888888;"> Shuffle Errors</span>
<span style="color: #888888;"> BAD_ID=0</span>
<span style="color: #888888;"> CONNECTION=0</span>
<span style="color: #888888;"> IO_ERROR=0</span>
<span style="color: #888888;"> WRONG_LENGTH=0</span>
<span style="color: #888888;"> WRONG_MAP=0</span>
<span style="color: #888888;"> WRONG_REDUCE=0</span>
<span style="color: #888888;"> File Input Format Counters </span>
<span style="color: #888888;"> Bytes Read=123</span>
<span style="color: #888888;"> File Output Format Counters </span>
<span style="color: #888888;"> Bytes Written=23</span>
<span style="color: #c65d09; font-weight: bold;">hadoop@637c83896b9d:~/hadoop-2.7.3$</span>
<span style="color: #c65d09; font-weight: bold;">hadoop@637c83896b9d:~/hadoop-2.7.3$</span>
<span style="color: #c65d09; font-weight: bold;">hadoop@637c83896b9d:~/hadoop-2.7.3$</span>
<span style="color: #c65d09; font-weight: bold;">hadoop@637c83896b9d:~/hadoop-2.7.3$</span>
<span style="color: #c65d09; font-weight: bold;">hadoop@637c83896b9d:~/hadoop-2.7.3$</span> ls share/hadoop/mapreduce/hadoop-mapreduce-examples-2.7.3.jar
<span style="color: #888888;">share/hadoop/mapreduce/hadoop-mapreduce-examples-2.7.3.jar</span>
<span style="color: #c65d09; font-weight: bold;">hadoop@637c83896b9d:~/hadoop-2.7.3$</span> cat output/*
<span style="color: #888888;">1 dfsadmin</span>
</pre>
</div>
* <span style="color: orange;">Orange text </span>highlight mean you close a file while a readahead request is in flight. (thanks this <a href="http://stackoverflow.com/questions/27253616/hadoop-warn-ebadf-bad-file-descriptor" target="_blank">forum</a>)<br />
<br />
<br />
Alternatively, test with unix grep command<br />
<br />
<!-- HTML generated using hilite.me --><br />
<div style="background: #ffffff; border-width: 0.1em 0.1em 0.1em 0.8em; border: solid gray; overflow: auto; padding: 0.2em 0.6em; width: auto;">
<pre style="line-height: 125%; margin: 0;"><span style="color: #c65d09; font-weight: bold;">hadoop@637c83896b9d:~/hadoop-2.7.3/input$</span> ls
<span style="color: #888888;">capacity-scheduler.xml core-site.xml hadoop-policy.xml hdfs-site.xml httpfs-site.xml kms-acls.xml kms-site.xml yarn-site.xml</span>
<span style="color: #c65d09; font-weight: bold;">hadoop@637c83896b9d:~/hadoop-2.7.3/input$</span> ls -lt
<span style="color: #888888;">total 48</span>
<span style="color: #888888;">-rw-r--r-- 1 hadoop hadoop 4436 Aug 28 10:30 capacity-scheduler.xml</span>
<span style="color: #888888;">-rw-r--r-- 1 hadoop hadoop 774 Aug 28 10:30 core-site.xml</span>
<span style="color: #888888;">-rw-r--r-- 1 hadoop hadoop 9683 Aug 28 10:30 hadoop-policy.xml</span>
<span style="color: #888888;">-rw-r--r-- 1 hadoop hadoop 775 Aug 28 10:30 hdfs-site.xml</span>
<span style="color: #888888;">-rw-r--r-- 1 hadoop hadoop 620 Aug 28 10:30 httpfs-site.xml</span>
<span style="color: #888888;">-rw-r--r-- 1 hadoop hadoop 3518 Aug 28 10:30 kms-acls.xml</span>
<span style="color: #888888;">-rw-r--r-- 1 hadoop hadoop 5511 Aug 28 10:30 kms-site.xml</span>
<span style="color: #888888;">-rw-r--r-- 1 hadoop hadoop 690 Aug 28 10:30 yarn-site.xml</span>
<span style="color: #c65d09; font-weight: bold;">hadoop@637c83896b9d:~/hadoop-2.7.3/input$</span> grep dfs *.xml
<span style="color: #888888;">hadoop-policy.xml: </span><span style="color: blue;">dfs</span><span style="color: #888888;">admin and mradmin commands to refresh the security policy in-effect.</span>
<span style="color: #c65d09; font-weight: bold;">hadoop@637c83896b9d:~/hadoop-2.7.3/input$</span> <span style="color: #007020;">cd</span> ..
<span style="color: #c65d09; font-weight: bold;">hadoop@637c83896b9d:~/hadoop-2.7.3$</span> <span style="color: #007020;">cd </span>output
<span style="color: #c65d09; font-weight: bold;">hadoop@637c83896b9d:~/hadoop-2.7.3/output$</span> ls -l
<span style="color: #888888;">total 4</span>
<span style="color: #888888;">-rw-r--r-- 1 hadoop hadoop 0 Aug 28 10:30 _SUCCESS</span>
<span style="color: #888888;">-rw-r--r-- 1 hadoop hadoop 11 Aug 28 10:30 part-r-00000</span>
<span style="color: #c65d09; font-weight: bold;">hadoop@637c83896b9d:~/hadoop-2.7.3/output$</span> cat *
<span style="color: #888888;">1 dfsadmin</span></pre>
</div>
<br />
<!-- HTML generated using hilite.me --><br />
<div style="background: #ffffff; border-width: 0.1em 0.1em 0.1em 0.8em; border: solid gray; overflow: auto; padding: 0.2em 0.6em; width: auto;">
<pre style="line-height: 125%; margin: 0;"><span style="color: #c65d09; font-weight: bold;">hadoop@637c83896b9d:~/hadoop-2.7.3$</span> sudo service ssh start
<span style="color: #888888;">[sudo] password for hadoop: </span>
<span style="color: #888888;">hadoop is not in the sudoers file. This incident will be reported.</span>
<span style="color: #c65d09; font-weight: bold;">hadoop@637c83896b9d:~/hadoop-2.7.3$</span> <span style="color: #007020;">exit</span>
<span style="color: #888888;">exit</span>
<span style="color: #c65d09; font-weight: bold;">ubuntu@node2:~$</span> docker <span style="color: #007020;">exec</span> -it hbase bash
<span style="color: #c65d09; font-weight: bold;">root@637c83896b9d:/#</span> service ssh start
<span style="color: #888888;"> * Starting OpenBSD Secure Shell server sshd [ OK ] </span>
<span style="color: #c65d09; font-weight: bold;">root@637c83896b9d:/#</span> <span style="color: #007020;">exit</span>
<span style="color: #888888;">exit</span>
<span style="color: #c65d09; font-weight: bold;">ubuntu@node2:~$</span> docker <span style="color: #007020;">exec</span> -it --user hadoop hbase bash
<span style="color: #c65d09; font-weight: bold;">hadoop@637c83896b9d:/$</span>
<span style="color: #c65d09; font-weight: bold;">hadoop@637c83896b9d:/$</span> ssh localhost
<span style="color: #888888;">The authenticity of host 'localhost (::1)' can't be established.</span>
<span style="color: #888888;">ECDSA key fingerprint is 04:0d:98:59:94:bc:94:83:1a:de:3d:ae:3d:9b:a0:20.</span>
<span style="color: #888888;">Are you sure you want to continue connecting (yes/no)? yes</span>
<span style="color: #888888;">Warning: Permanently added 'localhost' (ECDSA) to the list of known hosts.</span>
<span style="color: #888888;">hadoop@localhost's password: </span>
<span style="color: #888888;">Welcome to Ubuntu 14.04 LTS (GNU/Linux 3.13.0-92-generic x86_64)</span>
<span style="color: #888888;"> * Documentation: https://help.ubuntu.com/</span>
<span style="color: #888888;">The programs included with the Ubuntu system are free software;</span>
<span style="color: #888888;">the exact distribution terms for each program are described in the</span>
<span style="color: #888888;">individual files in /usr/share/doc/*/copyright.</span>
<span style="color: #888888;">Ubuntu comes with ABSOLUTELY NO WARRANTY, to the extent permitted by</span>
<span style="color: #888888;">applicable law.</span>
<span style="color: #c65d09; font-weight: bold;">hadoop@637c83896b9d:~$</span> <span style="color: #007020;">exit</span>
<span style="color: #888888;">logout</span>
<span style="color: #888888;">Connection to localhost closed.</span>
<span style="color: #c65d09; font-weight: bold;">hadoop@637c83896b9d:/$</span>
<span style="color: #c65d09; font-weight: bold;">hadoop@637c83896b9d:/$</span> ssh-keygen -t rsa -P <span style="background-color: #fff0f0;">''</span> -f ~/.ssh/id_rsa
<span style="color: #888888;">Generating public/private rsa key pair.</span>
<span style="color: #888888;">Your identification has been saved in /home/hadoop/.ssh/id_rsa.</span>
<span style="color: #888888;">Your public key has been saved in /home/hadoop/.ssh/id_rsa.pub.</span>
<span style="color: #888888;">The key fingerprint is:</span>
<span style="color: #888888;">b8:b5:59:7c:b5:1f:f8:af:ab:a3:74:d0:82:bf:a0:67 hadoop@637c83896b9d</span>
<span style="color: #888888;">The key's randomart image is:</span>
<span style="color: #888888;">+--[ RSA 2048]----+</span>
<span style="color: #888888;">| |</span>
<span style="color: #888888;">| |</span>
<span style="color: #888888;">| . |</span>
<span style="color: #888888;">| . o . ... |</span>
<span style="color: #888888;">| . S = o... |</span>
<span style="color: #888888;">| o = + ...|</span>
<span style="color: #888888;">| . + o . ..|</span>
<span style="color: #888888;">| .Eo o. .|</span>
<span style="color: #888888;">| .o o..ooo.|</span>
<span style="color: #888888;">+-----------------+</span>
<span style="color: #c65d09; font-weight: bold;">hadoop@637c83896b9d:/$</span> cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys
<span style="color: #c65d09; font-weight: bold;">hadoop@637c83896b9d:/$</span> chmod 0600 ~/.ssh/authorized_keys
<span style="color: #c65d09; font-weight: bold;">hadoop@637c83896b9d:/$</span>
<span style="color: #c65d09; font-weight: bold;">hadoop@637c83896b9d:/$</span> ssh localhost
<span style="color: #888888;">Welcome to Ubuntu 14.04.5 LTS (GNU/Linux 3.13.0-85-generic x86_64)</span>
<span style="color: #888888;"> * Documentation: https://help.ubuntu.com/</span>
<span style="color: #888888;">Last login: Sun Aug 28 11:03:32 2016 from localhost</span>
</pre>
</div>
<br />
<br />
Run a MapReduce job locally<br />
<br />
1) Format the file system<br />
<br />
<!-- HTML generated using hilite.me --><br />
<div style="background: #ffffff; border-width: 0.1em 0.1em 0.1em 0.8em; border: solid gray; overflow: auto; padding: 0.2em 0.6em; width: auto;">
<pre style="line-height: 125%; margin: 0;"><span style="color: #c65d09; font-weight: bold;">hadoop@637c83896b9d:~/hadoop-2.7.3$</span> bin/hdfs namenode -format
<span style="color: #888888;">16/08/28 11:09:29 INFO namenode.NameNode: STARTUP_MSG: </span>
<span style="color: #888888;">/************************************************************</span>
<span style="color: #888888;">STARTUP_MSG: Starting NameNode</span>
<span style="color: #888888;">STARTUP_MSG: host = hbase/172.29.5.1</span>
<span style="color: #888888;">STARTUP_MSG: args = [-format]</span>
<span style="color: #888888;">STARTUP_MSG: version = 2.7.3</span>
<span style="color: #888888;">STARTUP_MSG: classpath = /home/hadoop/hadoop-2.7.3/etc/hadoop:/home/hadoop/hadoop-2.7.3/share/hadoop/common/lib/paranamer-2.3.jar:/home/hadoop/hadoop-2.7.3/share/hadoop/common/lib/api-util-1.0.0-M20.jar:/home/hadoop/hadoop-2.7.3/share/hadoop/common/lib/jersey-json-1.9.jar:/home/hadoop/hadoop-2.7.3/share/hadoop/common/lib/java-xmlbuilder-0.4.jar:/home/hadoop/hadoop-2.7.3/share/hadoop/common/lib/jackson-mapper-asl-1.9.13.jar:/home/hadoop/hadoop-2.7.3/share/hadoop/common/lib/asm-3.2.jar:/home/hadoop/hadoop-2.7.3/share/hadoop/common/lib/commons-httpclient-3.1.jar:/home/hadoop/hadoop-2.7.3/share/hadoop/common/lib/hadoop-auth-2.7.3.jar:/home/hadoop/hadoop-2.7.3/share/hadoop/common/lib/apacheds-kerberos-codec-2.0.0-M15.jar:/home/hadoop/hadoop-2.7.3/share/hadoop/common/lib/commons-net-3.1.jar:/home/hadoop/hadoop-2.7.3/share/hadoop/common/lib/commons-compress-1.4.1.jar:/home/hadoop/hadoop-2.7.3/share/hadoop/common/lib/curator-framework-2.7.1.jar:/home/hadoop/hadoop-2.7.3/share/hadoop/common/lib/hamcrest-core-1.3.jar:/home/hadoop/hadoop-2.7.3/share/hadoop/common/lib/apacheds-i18n-2.0.0-M15.jar:/home/hadoop/hadoop-2.7.3/share/hadoop/common/lib/snappy-java-1.0.4.1.jar:/home/hadoop/hadoop-2.7.3/share/hadoop/common/lib/jetty-6.1.26.jar:/home/hadoop/hadoop-2.7.3/share/hadoop/common/lib/avro-1.7.4.jar:/home/hadoop/hadoop-2.7.3/share/hadoop/common/lib/commons-codec-1.4.jar:/home/hadoop/hadoop-2.7.3/share/hadoop/common/lib/commons-lang-2.6.jar:/home/hadoop/hadoop-2.7.3/share/hadoop/common/lib/jets3t-0.9.0.jar:/home/hadoop/hadoop-2.7.3/share/hadoop/common/lib/slf4j-log4j12-1.7.10.jar:/home/hadoop/hadoop-2.7.3/share/hadoop/common/lib/log4j-1.2.17.jar:/home/hadoop/hadoop-2.7.3/share/hadoop/common/lib/netty-3.6.2.Final.jar:/home/hadoop/hadoop-2.7.3/share/hadoop/common/lib/jersey-server-1.9.jar:/home/hadoop/hadoop-2.7.3/share/hadoop/common/lib/commons-cli-1.2.jar:/home/hadoop/hadoop-2.7.3/share/hadoop/common/lib/hadoop-annotations-2.7.3.jar:/home/hadoop/hadoop-2.7.3/share/hadoop/common/lib/protobuf-java-2.5.0.jar:/home/hadoop/hadoop-2.7.3/share/hadoop/common/lib/jetty-util-6.1.26.jar:/home/hadoop/hadoop-2.7.3/share/hadoop/common/lib/jettison-1.1.jar:/home/hadoop/hadoop-2.7.3/share/hadoop/common/lib/zookeeper-3.4.6.jar:/home/hadoop/hadoop-2.7.3/share/hadoop/common/lib/jaxb-api-2.2.2.jar:/home/hadoop/hadoop-2.7.3/share/hadoop/common/lib/stax-api-1.0-2.jar:/home/hadoop/hadoop-2.7.3/share/hadoop/common/lib/commons-digester-1.8.jar:/home/hadoop/hadoop-2.7.3/share/hadoop/common/lib/jackson-core-asl-1.9.13.jar:/home/hadoop/hadoop-2.7.3/share/hadoop/common/lib/jersey-core-1.9.jar:/home/hadoop/hadoop-2.7.3/share/hadoop/common/lib/htrace-core-3.1.0-incubating.jar:/home/hadoop/hadoop-2.7.3/share/hadoop/common/lib/commons-beanutils-core-1.8.0.jar:/home/hadoop/hadoop-2.7.3/share/hadoop/common/lib/commons-math3-3.1.1.jar:/home/hadoop/hadoop-2.7.3/share/hadoop/common/lib/jsch-0.1.42.jar:/home/hadoop/hadoop-2.7.3/share/hadoop/common/lib/httpclient-4.2.5.jar:/home/hadoop/hadoop-2.7.3/share/hadoop/common/lib/jackson-xc-1.9.13.jar:/home/hadoop/hadoop-2.7.3/share/hadoop/common/lib/httpcore-4.2.5.jar:/home/hadoop/hadoop-2.7.3/share/hadoop/common/lib/junit-4.11.jar:/home/hadoop/hadoop-2.7.3/share/hadoop/common/lib/activation-1.1.jar:/home/hadoop/hadoop-2.7.3/share/hadoop/common/lib/xz-1.0.jar:/home/hadoop/hadoop-2.7.3/share/hadoop/common/lib/commons-configuration-1.6.jar:/home/hadoop/hadoop-2.7.3/share/hadoop/common/lib/xmlenc-0.52.jar:/home/hadoop/hadoop-2.7.3/share/hadoop/common/lib/commons-logging-1.1.3.jar:/home/hadoop/hadoop-2.7.3/share/hadoop/common/lib/gson-2.2.4.jar:/home/hadoop/hadoop-2.7.3/share/hadoop/common/lib/api-asn1-api-1.0.0-M20.jar:/home/hadoop/hadoop-2.7.3/share/hadoop/common/lib/curator-recipes-2.7.1.jar:/home/hadoop/hadoop-2.7.3/share/hadoop/common/lib/curator-client-2.7.1.jar:/home/hadoop/hadoop-2.7.3/share/hadoop/common/lib/slf4j-api-1.7.10.jar:/home/hadoop/hadoop-2.7.3/share/hadoop/common/lib/commons-collections-3.2.2.jar:/home/hadoop/hadoop-2.7.3/share/hadoop/common/lib/jsr305-3.0.0.jar:/home/hadoop/hadoop-2.7.3/share/hadoop/common/lib/commons-beanutils-1.7.0.jar:/home/hadoop/hadoop-2.7.3/share/hadoop/common/lib/jsp-api-2.1.jar:/home/hadoop/hadoop-2.7.3/share/hadoop/common/lib/commons-io-2.4.jar:/home/hadoop/hadoop-2.7.3/share/hadoop/common/lib/jaxb-impl-2.2.3-1.jar:/home/hadoop/hadoop-2.7.3/share/hadoop/common/lib/servlet-api-2.5.jar:/home/hadoop/hadoop-2.7.3/share/hadoop/common/lib/mockito-all-1.8.5.jar:/home/hadoop/hadoop-2.7.3/share/hadoop/common/lib/jackson-jaxrs-1.9.13.jar:/home/hadoop/hadoop-2.7.3/share/hadoop/common/lib/guava-11.0.2.jar:/home/hadoop/hadoop-2.7.3/share/hadoop/common/hadoop-nfs-2.7.3.jar:/home/hadoop/hadoop-2.7.3/share/hadoop/common/hadoop-common-2.7.3.jar:/home/hadoop/hadoop-2.7.3/share/hadoop/common/hadoop-common-2.7.3-tests.jar:/home/hadoop/hadoop-2.7.3/share/hadoop/hdfs:/home/hadoop/hadoop-2.7.3/share/hadoop/hdfs/lib/jackson-mapper-asl-1.9.13.jar:/home/hadoop/hadoop-2.7.3/share/hadoop/hdfs/lib/asm-3.2.jar:/home/hadoop/hadoop-2.7.3/share/hadoop/hdfs/lib/netty-all-4.0.23.Final.jar:/home/hadoop/hadoop-2.7.3/share/hadoop/hdfs/lib/jetty-6.1.26.jar:/home/hadoop/hadoop-2.7.3/share/hadoop/hdfs/lib/commons-codec-1.4.jar:/home/hadoop/hadoop-2.7.3/share/hadoop/hdfs/lib/commons-lang-2.6.jar:/home/hadoop/hadoop-2.7.3/share/hadoop/hdfs/lib/log4j-1.2.17.jar:/home/hadoop/hadoop-2.7.3/share/hadoop/hdfs/lib/netty-3.6.2.Final.jar:/home/hadoop/hadoop-2.7.3/share/hadoop/hdfs/lib/jersey-server-1.9.jar:/home/hadoop/hadoop-2.7.3/share/hadoop/hdfs/lib/commons-cli-1.2.jar:/home/hadoop/hadoop-2.7.3/share/hadoop/hdfs/lib/protobuf-java-2.5.0.jar:/home/hadoop/hadoop-2.7.3/share/hadoop/hdfs/lib/jetty-util-6.1.26.jar:/home/hadoop/hadoop-2.7.3/share/hadoop/hdfs/lib/xml-apis-1.3.04.jar:/home/hadoop/hadoop-2.7.3/share/hadoop/hdfs/lib/jackson-core-asl-1.9.13.jar:/home/hadoop/hadoop-2.7.3/share/hadoop/hdfs/lib/commons-daemon-1.0.13.jar:/home/hadoop/hadoop-2.7.3/share/hadoop/hdfs/lib/xercesImpl-2.9.1.jar:/home/hadoop/hadoop-2.7.3/share/hadoop/hdfs/lib/jersey-core-1.9.jar:/home/hadoop/hadoop-2.7.3/share/hadoop/hdfs/lib/htrace-core-3.1.0-incubating.jar:/home/hadoop/hadoop-2.7.3/share/hadoop/hdfs/lib/xmlenc-0.52.jar:/home/hadoop/hadoop-2.7.3/share/hadoop/hdfs/lib/commons-logging-1.1.3.jar:/home/hadoop/hadoop-2.7.3/share/hadoop/hdfs/lib/jsr305-3.0.0.jar:/home/hadoop/hadoop-2.7.3/share/hadoop/hdfs/lib/commons-io-2.4.jar:/home/hadoop/hadoop-2.7.3/share/hadoop/hdfs/lib/servlet-api-2.5.jar:/home/hadoop/hadoop-2.7.3/share/hadoop/hdfs/lib/leveldbjni-all-1.8.jar:/home/hadoop/hadoop-2.7.3/share/hadoop/hdfs/lib/guava-11.0.2.jar:/home/hadoop/hadoop-2.7.3/share/hadoop/hdfs/hadoop-hdfs-2.7.3.jar:/home/hadoop/hadoop-2.7.3/share/hadoop/hdfs/hadoop-hdfs-2.7.3-tests.jar:/home/hadoop/hadoop-2.7.3/share/hadoop/hdfs/hadoop-hdfs-nfs-2.7.3.jar:/home/hadoop/hadoop-2.7.3/share/hadoop/yarn/lib/jersey-client-1.9.jar:/home/hadoop/hadoop-2.7.3/share/hadoop/yarn/lib/guice-servlet-3.0.jar:/home/hadoop/hadoop-2.7.3/share/hadoop/yarn/lib/jersey-json-1.9.jar:/home/hadoop/hadoop-2.7.3/share/hadoop/yarn/lib/aopalliance-1.0.jar:/home/hadoop/hadoop-2.7.3/share/hadoop/yarn/lib/jackson-mapper-asl-1.9.13.jar:/home/hadoop/hadoop-2.7.3/share/hadoop/yarn/lib/asm-3.2.jar:/home/hadoop/hadoop-2.7.3/share/hadoop/yarn/lib/commons-compress-1.4.1.jar:/home/hadoop/hadoop-2.7.3/share/hadoop/yarn/lib/jetty-6.1.26.jar:/home/hadoop/hadoop-2.7.3/share/hadoop/yarn/lib/commons-codec-1.4.jar:/home/hadoop/hadoop-2.7.3/share/hadoop/yarn/lib/commons-lang-2.6.jar:/home/hadoop/hadoop-2.7.3/share/hadoop/yarn/lib/log4j-1.2.17.jar:/home/hadoop/hadoop-2.7.3/share/hadoop/yarn/lib/jersey-guice-1.9.jar:/home/hadoop/hadoop-2.7.3/share/hadoop/yarn/lib/netty-3.6.2.Final.jar:/home/hadoop/hadoop-2.7.3/share/hadoop/yarn/lib/jersey-server-1.9.jar:/home/hadoop/hadoop-2.7.3/share/hadoop/yarn/lib/commons-cli-1.2.jar:/home/hadoop/hadoop-2.7.3/share/hadoop/yarn/lib/protobuf-java-2.5.0.jar:/home/hadoop/hadoop-2.7.3/share/hadoop/yarn/lib/jetty-util-6.1.26.jar:/home/hadoop/hadoop-2.7.3/share/hadoop/yarn/lib/jettison-1.1.jar:/home/hadoop/hadoop-2.7.3/share/hadoop/yarn/lib/zookeeper-3.4.6.jar:/home/hadoop/hadoop-2.7.3/share/hadoop/yarn/lib/jaxb-api-2.2.2.jar:/home/hadoop/hadoop-2.7.3/share/hadoop/yarn/lib/stax-api-1.0-2.jar:/home/hadoop/hadoop-2.7.3/share/hadoop/yarn/lib/jackson-core-asl-1.9.13.jar:/home/hadoop/hadoop-2.7.3/share/hadoop/yarn/lib/jersey-core-1.9.jar:/home/hadoop/hadoop-2.7.3/share/hadoop/yarn/lib/guice-3.0.jar:/home/hadoop/hadoop-2.7.3/share/hadoop/yarn/lib/javax.inject-1.jar:/home/hadoop/hadoop-2.7.3/share/hadoop/yarn/lib/jackson-xc-1.9.13.jar:/home/hadoop/hadoop-2.7.3/share/hadoop/yarn/lib/zookeeper-3.4.6-tests.jar:/home/hadoop/hadoop-2.7.3/share/hadoop/yarn/lib/activation-1.1.jar:/home/hadoop/hadoop-2.7.3/share/hadoop/yarn/lib/xz-1.0.jar:/home/hadoop/hadoop-2.7.3/share/hadoop/yarn/lib/commons-logging-1.1.3.jar:/home/hadoop/hadoop-2.7.3/share/hadoop/yarn/lib/commons-collections-3.2.2.jar:/home/hadoop/hadoop-2.7.3/share/hadoop/yarn/lib/jsr305-3.0.0.jar:/home/hadoop/hadoop-2.7.3/share/hadoop/yarn/lib/commons-io-2.4.jar:/home/hadoop/hadoop-2.7.3/share/hadoop/yarn/lib/jaxb-impl-2.2.3-1.jar:/home/hadoop/hadoop-2.7.3/share/hadoop/yarn/lib/servlet-api-2.5.jar:/home/hadoop/hadoop-2.7.3/share/hadoop/yarn/lib/leveldbjni-all-1.8.jar:/home/hadoop/hadoop-2.7.3/share/hadoop/yarn/lib/jackson-jaxrs-1.9.13.jar:/home/hadoop/hadoop-2.7.3/share/hadoop/yarn/lib/guava-11.0.2.jar:/home/hadoop/hadoop-2.7.3/share/hadoop/yarn/hadoop-yarn-applications-unmanaged-am-launcher-2.7.3.jar:/home/hadoop/hadoop-2.7.3/share/hadoop/yarn/hadoop-yarn-server-resourcemanager-2.7.3.jar:/home/hadoop/hadoop-2.7.3/share/hadoop/yarn/hadoop-yarn-client-2.7.3.jar:/home/hadoop/hadoop-2.7.3/share/hadoop/yarn/hadoop-yarn-applications-distributedshell-2.7.3.jar:/home/hadoop/hadoop-2.7.3/share/hadoop/yarn/hadoop-yarn-server-web-proxy-2.7.3.jar:/home/hadoop/hadoop-2.7.3/share/hadoop/yarn/hadoop-yarn-server-common-2.7.3.jar:/home/hadoop/hadoop-2.7.3/share/hadoop/yarn/hadoop-yarn-server-nodemanager-2.7.3.jar:/home/hadoop/hadoop-2.7.3/share/hadoop/yarn/hadoop-yarn-registry-2.7.3.jar:/home/hadoop/hadoop-2.7.3/share/hadoop/yarn/hadoop-yarn-api-2.7.3.jar:/home/hadoop/hadoop-2.7.3/share/hadoop/yarn/hadoop-yarn-server-sharedcachemanager-2.7.3.jar:/home/hadoop/hadoop-2.7.3/share/hadoop/yarn/hadoop-yarn-server-tests-2.7.3.jar:/home/hadoop/hadoop-2.7.3/share/hadoop/yarn/hadoop-yarn-common-2.7.3.jar:/home/hadoop/hadoop-2.7.3/share/hadoop/yarn/hadoop-yarn-server-applicationhistoryservice-2.7.3.jar:/home/hadoop/hadoop-2.7.3/share/hadoop/mapreduce/lib/paranamer-2.3.jar:/home/hadoop/hadoop-2.7.3/share/hadoop/mapreduce/lib/guice-servlet-3.0.jar:/home/hadoop/hadoop-2.7.3/share/hadoop/mapreduce/lib/aopalliance-1.0.jar:/home/hadoop/hadoop-2.7.3/share/hadoop/mapreduce/lib/jackson-mapper-asl-1.9.13.jar:/home/hadoop/hadoop-2.7.3/share/hadoop/mapreduce/lib/asm-3.2.jar:/home/hadoop/hadoop-2.7.3/share/hadoop/mapreduce/lib/commons-compress-1.4.1.jar:/home/hadoop/hadoop-2.7.3/share/hadoop/mapreduce/lib/hamcrest-core-1.3.jar:/home/hadoop/hadoop-2.7.3/share/hadoop/mapreduce/lib/snappy-java-1.0.4.1.jar:/home/hadoop/hadoop-2.7.3/share/hadoop/mapreduce/lib/avro-1.7.4.jar:/home/hadoop/hadoop-2.7.3/share/hadoop/mapreduce/lib/log4j-1.2.17.jar:/home/hadoop/hadoop-2.7.3/share/hadoop/mapreduce/lib/jersey-guice-1.9.jar:/home/hadoop/hadoop-2.7.3/share/hadoop/mapreduce/lib/netty-3.6.2.Final.jar:/home/hadoop/hadoop-2.7.3/share/hadoop/mapreduce/lib/jersey-server-1.9.jar:/home/hadoop/hadoop-2.7.3/share/hadoop/mapreduce/lib/hadoop-annotations-2.7.3.jar:/home/hadoop/hadoop-2.7.3/share/hadoop/mapreduce/lib/protobuf-java-2.5.0.jar:/home/hadoop/hadoop-2.7.3/share/hadoop/mapreduce/lib/jackson-core-asl-1.9.13.jar:/home/hadoop/hadoop-2.7.3/share/hadoop/mapreduce/lib/jersey-core-1.9.jar:/home/hadoop/hadoop-2.7.3/share/hadoop/mapreduce/lib/guice-3.0.jar:/home/hadoop/hadoop-2.7.3/share/hadoop/mapreduce/lib/javax.inject-1.jar:/home/hadoop/hadoop-2.7.3/share/hadoop/mapreduce/lib/junit-4.11.jar:/home/hadoop/hadoop-2.7.3/share/hadoop/mapreduce/lib/xz-1.0.jar:/home/hadoop/hadoop-2.7.3/share/hadoop/mapreduce/lib/commons-io-2.4.jar:/home/hadoop/hadoop-2.7.3/share/hadoop/mapreduce/lib/leveldbjni-all-1.8.jar:/home/hadoop/hadoop-2.7.3/share/hadoop/mapreduce/hadoop-mapreduce-client-hs-2.7.3.jar:/home/hadoop/hadoop-2.7.3/share/hadoop/mapreduce/hadoop-mapreduce-client-common-2.7.3.jar:/home/hadoop/hadoop-2.7.3/share/hadoop/mapreduce/hadoop-mapreduce-client-core-2.7.3.jar:/home/hadoop/hadoop-2.7.3/share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-2.7.3.jar:/home/hadoop/hadoop-2.7.3/share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-2.7.3-tests.jar:/home/hadoop/hadoop-2.7.3/share/hadoop/mapreduce/hadoop-mapreduce-client-shuffle-2.7.3.jar:/home/hadoop/hadoop-2.7.3/share/hadoop/mapreduce/hadoop-mapreduce-examples-2.7.3.jar:/home/hadoop/hadoop-2.7.3/share/hadoop/mapreduce/hadoop-mapreduce-client-hs-plugins-2.7.3.jar:/home/hadoop/hadoop-2.7.3/share/hadoop/mapreduce/hadoop-mapreduce-client-app-2.7.3.jar:/contrib/capacity-scheduler/*.jar</span>
<span style="color: #888888;">STARTUP_MSG: build = https://git-wip-us.apache.org/repos/asf/hadoop.git -r baa91f7c6bc9cb92be5982de4719c1c8af91ccff; compiled by 'root' on 2016-08-18T01:41Z</span>
<span style="color: #888888;">STARTUP_MSG: java = 1.8.0_101</span>
<span style="color: #888888;">************************************************************/</span>
<span style="color: #888888;">16/08/28 11:09:29 INFO namenode.NameNode: registered UNIX signal handlers for [TERM, HUP, INT]</span>
<span style="color: #888888;">16/08/28 11:09:29 INFO namenode.NameNode: createNameNode [-format]</span>
<span style="color: #888888;">Formatting using clusterid: CID-83a59c31-a355-4323-9334-d14989b04a59</span>
<span style="color: #888888;">16/08/28 11:09:29 INFO namenode.FSNamesystem: No KeyProvider found.</span>
<span style="color: #888888;">16/08/28 11:09:29 INFO namenode.FSNamesystem: fsLock is fair:true</span>
<span style="color: #888888;">16/08/28 11:09:29 INFO blockmanagement.DatanodeManager: dfs.block.invalidate.limit=1000</span>
<span style="color: #888888;">16/08/28 11:09:29 INFO blockmanagement.DatanodeManager: dfs.namenode.datanode.registration.ip-hostname-check=true</span>
<span style="color: #888888;">16/08/28 11:09:29 INFO blockmanagement.BlockManager: dfs.namenode.startup.delay.block.deletion.sec is set to 000:00:00:00.000</span>
<span style="color: #888888;">16/08/28 11:09:29 INFO blockmanagement.BlockManager: The block deletion will start around 2016 Aug 28 11:09:29</span>
<span style="color: #888888;">16/08/28 11:09:29 INFO util.GSet: Computing capacity for map BlocksMap</span>
<span style="color: #888888;">16/08/28 11:09:29 INFO util.GSet: VM type = 64-bit</span>
<span style="color: #888888;">16/08/28 11:09:29 INFO util.GSet: 2.0% max memory 889 MB = 17.8 MB</span>
<span style="color: #888888;">16/08/28 11:09:29 INFO util.GSet: capacity = 2^21 = 2097152 entries</span>
<span style="color: #888888;">16/08/28 11:09:29 INFO blockmanagement.BlockManager: dfs.block.access.token.enable=false</span>
<span style="color: #888888;">16/08/28 11:09:29 INFO blockmanagement.BlockManager: defaultReplication = 1</span>
<span style="color: #888888;">16/08/28 11:09:29 INFO blockmanagement.BlockManager: maxReplication = 512</span>
<span style="color: #888888;">16/08/28 11:09:29 INFO blockmanagement.BlockManager: minReplication = 1</span>
<span style="color: #888888;">16/08/28 11:09:29 INFO blockmanagement.BlockManager: maxReplicationStreams = 2</span>
<span style="color: #888888;">16/08/28 11:09:29 INFO blockmanagement.BlockManager: replicationRecheckInterval = 3000</span>
<span style="color: #888888;">16/08/28 11:09:29 INFO blockmanagement.BlockManager: encryptDataTransfer = false</span>
<span style="color: #888888;">16/08/28 11:09:29 INFO blockmanagement.BlockManager: maxNumBlocksToLog = 1000</span>
<span style="color: #888888;">16/08/28 11:09:29 INFO namenode.FSNamesystem: fsOwner = hadoop (auth:SIMPLE)</span>
<span style="color: #888888;">16/08/28 11:09:29 INFO namenode.FSNamesystem: supergroup = supergroup</span>
<span style="color: #888888;">16/08/28 11:09:29 INFO namenode.FSNamesystem: isPermissionEnabled = true</span>
<span style="color: #888888;">16/08/28 11:09:29 INFO namenode.FSNamesystem: HA Enabled: false</span>
<span style="color: #888888;">16/08/28 11:09:29 INFO namenode.FSNamesystem: Append Enabled: true</span>
<span style="color: #888888;">16/08/28 11:09:29 INFO util.GSet: Computing capacity for map INodeMap</span>
<span style="color: #888888;">16/08/28 11:09:29 INFO util.GSet: VM type = 64-bit</span>
<span style="color: #888888;">16/08/28 11:09:29 INFO util.GSet: 1.0% max memory 889 MB = 8.9 MB</span>
<span style="color: #888888;">16/08/28 11:09:29 INFO util.GSet: capacity = 2^20 = 1048576 entries</span>
<span style="color: #888888;">16/08/28 11:09:29 INFO namenode.FSDirectory: ACLs enabled? false</span>
<span style="color: #888888;">16/08/28 11:09:29 INFO namenode.FSDirectory: XAttrs enabled? true</span>
<span style="color: #888888;">16/08/28 11:09:29 INFO namenode.FSDirectory: Maximum size of an xattr: 16384</span>
<span style="color: #888888;">16/08/28 11:09:29 INFO namenode.NameNode: Caching file names occuring more than 10 times</span>
<span style="color: #888888;">16/08/28 11:09:29 INFO util.GSet: Computing capacity for map cachedBlocks</span>
<span style="color: #888888;">16/08/28 11:09:29 INFO util.GSet: VM type = 64-bit</span>
<span style="color: #888888;">16/08/28 11:09:29 INFO util.GSet: 0.25% max memory 889 MB = 2.2 MB</span>
<span style="color: #888888;">16/08/28 11:09:29 INFO util.GSet: capacity = 2^18 = 262144 entries</span>
<span style="color: #888888;">16/08/28 11:09:29 INFO namenode.FSNamesystem: dfs.namenode.safemode.threshold-pct = 0.9990000128746033</span>
<span style="color: #888888;">16/08/28 11:09:29 INFO namenode.FSNamesystem: dfs.namenode.safemode.min.datanodes = 0</span>
<span style="color: #888888;">16/08/28 11:09:29 INFO namenode.FSNamesystem: dfs.namenode.safemode.extension = 30000</span>
<span style="color: #888888;">16/08/28 11:09:29 INFO metrics.TopMetrics: NNTop conf: dfs.namenode.top.window.num.buckets = 10</span>
<span style="color: #888888;">16/08/28 11:09:29 INFO metrics.TopMetrics: NNTop conf: dfs.namenode.top.num.users = 10</span>
<span style="color: #888888;">16/08/28 11:09:29 INFO metrics.TopMetrics: NNTop conf: dfs.namenode.top.windows.minutes = 1,5,25</span>
<span style="color: #888888;">16/08/28 11:09:29 INFO namenode.FSNamesystem: Retry cache on namenode is enabled</span>
<span style="color: #888888;">16/08/28 11:09:29 INFO namenode.FSNamesystem: Retry cache will use 0.03 of total heap and retry cache entry expiry time is 600000 millis</span>
<span style="color: #888888;">16/08/28 11:09:29 INFO util.GSet: Computing capacity for map NameNodeRetryCache</span>
<span style="color: #888888;">16/08/28 11:09:29 INFO util.GSet: VM type = 64-bit</span>
<span style="color: #888888;">16/08/28 11:09:29 INFO util.GSet: 0.029999999329447746% max memory 889 MB = 273.1 KB</span>
<span style="color: #888888;">16/08/28 11:09:29 INFO util.GSet: capacity = 2^15 = 32768 entries</span>
<span style="color: #888888;">16/08/28 11:09:29 INFO namenode.FSImage: Allocated new BlockPoolId: BP-1423794091-172.29.5.1-1472382569816</span>
<span style="color: #888888;">16/08/28 11:09:30 INFO common.Storage: Storage directory /tmp/hadoop-hadoop/dfs/name has been successfully formatted.</span>
<span style="color: #888888;">16/08/28 11:09:30 INFO namenode.FSImageFormatProtobuf: Saving image file /tmp/hadoop-hadoop/dfs/name/current/fsimage.ckpt_0000000000000000000 using no compression</span>
<span style="color: #888888;">16/08/28 11:09:30 INFO namenode.FSImageFormatProtobuf: Image file /tmp/hadoop-hadoop/dfs/name/current/fsimage.ckpt_0000000000000000000 of size 353 bytes saved in 0 seconds.</span>
<span style="color: #888888;">16/08/28 11:09:30 INFO namenode.NNStorageRetentionManager: Going to retain 1 images with txid >= 0</span>
<span style="color: #888888;">16/08/28 11:09:30 INFO util.ExitUtil: Exiting with status 0</span>
<span style="color: #888888;">16/08/28 11:09:30 INFO namenode.NameNode: SHUTDOWN_MSG: </span>
<span style="color: #888888;">/************************************************************</span>
<span style="color: #888888;">SHUTDOWN_MSG: Shutting down NameNode at hbase/172.29.5.1</span>
<span style="color: #888888;">************************************************************/</span>
<span style="color: #c65d09; font-weight: bold;">hadoop@637c83896b9d:~/hadoop-2.7.3$</span>
</pre>
</div>
<br />
<br />
2) Start NameNode daemon and DataNode daemon<br />
<br />
Attention : Must set JAVA_HOME in hadoop env files to avoid startuo problems.<br />
<br />
<!-- HTML generated using hilite.me --><br />
<div style="background: #ffffff; border-width: 0.1em 0.1em 0.1em 0.8em; border: solid gray; overflow: auto; padding: 0.2em 0.6em; width: auto;">
<pre style="line-height: 125%; margin: 0;"><span style="color: #c65d09; font-weight: bold;">hadoop@637c83896b9d:~/hadoop-2.7.3$</span> grep JAVA_HOME etc/hadoop/hadoop-env.sh
<span style="color: #c65d09; font-weight: bold;">#</span> The only required environment variable is JAVA_HOME. All others are
<span style="color: #c65d09; font-weight: bold;">#</span> <span style="color: #007020;">set </span>JAVA_HOME in this file, so that it is correctly defined on
<span style="color: #c65d09; font-weight: bold;">#</span><span style="color: #007020;">export </span><span style="color: #996633;">JAVA_HOME</span><span style="color: #333333;">=</span><span style="color: #008800; font-weight: bold;">${</span><span style="color: #996633;">JAVA_HOME</span><span style="color: #008800; font-weight: bold;">}</span>
<span style="color: #888888;">export JAVA_HOME=/usr/lib/jvm/java-8-oracle</span>
<span style="color: #c65d09; font-weight: bold;">hadoop@637c83896b9d:~/hadoop-2.7.3$</span> sbin/start-dfs.sh
<span style="color: #888888;">Starting namenodes on [localhost]</span>
<span style="color: #888888;">localhost: starting namenode, logging to /home/hadoop/hadoop-2.7.3/logs/hadoop-hadoop-namenode-637c83896b9d.out</span>
<span style="color: #888888;">localhost: starting datanode, logging to /home/hadoop/hadoop-2.7.3/logs/hadoop-hadoop-datanode-637c83896b9d.out</span>
<span style="color: #888888;">Starting secondary namenodes [0.0.0.0]</span>
<span style="color: #888888;">0.0.0.0: starting secondarynamenode, logging to /home/hadoop/hadoop-2.7.3/logs/hadoop-hadoop-secondarynamenode-637c83896b9d.out</span>
<span style="color: #c65d09; font-weight: bold;">hadoop@637c83896b9d:~/hadoop-2.7.3$</span>
</pre>
</div>
<br />
Can browse to NameNode - http://localhost:50070/<br />
<div>
<br /></div>
<!-- HTML generated using hilite.me --><br />
<div style="background: #ffffff; border-width: 0.1em 0.1em 0.1em 0.8em; border: solid gray; overflow: auto; padding: 0.2em 0.6em; width: auto;">
<pre style="line-height: 125%; margin: 0;"><span style="color: #c65d09; font-weight: bold;">hadoop@637c83896b9d:~/hadoop-2.7.3$</span>
<span style="color: #c65d09; font-weight: bold;">hadoop@637c83896b9d:~/hadoop-2.7.3$</span> curl http://localhost:50070/
<span style="color: #888888;"><!--</span>
<span style="color: #888888;"> Licensed to the Apache Software Foundation (ASF) under one or more</span>
<span style="color: #888888;"> contributor license agreements. See the NOTICE file distributed with</span>
<span style="color: #888888;"> this work for additional information regarding copyright ownership.</span>
<span style="color: #888888;"> The ASF licenses this file to You under the Apache License, Version 2.0</span>
<span style="color: #888888;"> (the "License"); you may not use this file except in compliance with</span>
<span style="color: #888888;"> the License. You may obtain a copy of the License at</span>
<span style="color: #888888;"> http://www.apache.org/licenses/LICENSE-2.0</span>
<span style="color: #888888;"> Unless required by applicable law or agreed to in writing, software</span>
<span style="color: #888888;"> distributed under the License is distributed on an "AS IS" BASIS,</span>
<span style="color: #888888;"> WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.</span>
<span style="color: #888888;"> See the License for the specific language governing permissions and</span>
<span style="color: #888888;"> limitations under the License.</span>
<span style="color: #888888;">--></span>
<span style="color: #888888;"><!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN"</span>
<span style="color: #888888;"> "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd"></span>
<span style="color: #888888;"><html xmlns="http://www.w3.org/1999/xhtml"></span>
<span style="color: #888888;"><head></span>
<span style="color: #888888;"><meta http-equiv="REFRESH" content="0;url=dfshealth.html" /></span>
<span style="color: #888888;"><title>Hadoop Administration</title></span>
<span style="color: #888888;"></head></span>
<span style="color: #888888;"></html></span>
<span style="color: #c65d09; font-weight: bold;">hadoop@637c83896b9d:~/hadoop-2.7.3$</span>
</pre>
</div>
<br />
<!-- HTML generated using hilite.me --><br />
<div style="background: #ffffff; border-width: 0.1em 0.1em 0.1em 0.8em; border: solid gray; overflow: auto; padding: 0.2em 0.6em; width: auto;">
<pre style="line-height: 125%; margin: 0;"><span style="color: #c65d09; font-weight: bold;">hadoop@637c83896b9d:~/hadoop-2.7.3$</span> bin/hdfs dfs -mkdir /user
<span style="color: #c65d09; font-weight: bold;">hadoop@637c83896b9d:~/hadoop-2.7.3$</span>
<span style="color: #c65d09; font-weight: bold;">hadoop@637c83896b9d:~/hadoop-2.7.3$</span>
<span style="color: #c65d09; font-weight: bold;">hadoop@637c83896b9d:~/hadoop-2.7.3$</span>
<span style="color: #c65d09; font-weight: bold;">hadoop@637c83896b9d:~/hadoop-2.7.3$</span> bin/hdfs dfs -mkdir /user/hadoop
<span style="color: #c65d09; font-weight: bold;">hadoop@637c83896b9d:~/hadoop-2.7.3$</span>
<span style="color: #c65d09; font-weight: bold;">hadoop@637c83896b9d:~/hadoop-2.7.3$</span>
<span style="color: #c65d09; font-weight: bold;">hadoop@637c83896b9d:~/hadoop-2.7.3$</span>
<span style="color: #c65d09; font-weight: bold;">hadoop@637c83896b9d:~/hadoop-2.7.3$</span> bin/hdfs dfs -put etc/hadoop input
<span style="color: #c65d09; font-weight: bold;">hadoop@637c83896b9d:~/hadoop-2.7.3$</span>
<span style="color: #c65d09; font-weight: bold;">hadoop@637c83896b9d:~/hadoop-2.7.3$</span>
<span style="color: #c65d09; font-weight: bold;">hadoop@637c83896b9d:~/hadoop-2.7.3$</span>
<span style="color: #c65d09; font-weight: bold;">hadoop@637c83896b9d:~/hadoop-2.7.3$</span> bin/hadoop jar share/hadoop/mapreduce/hadoop-mapreduce-examples-2.7.3.jar grep input output <span style="background-color: #fff0f0;">'dfs[a-z.]+'</span>
<span style="color: #888888;">16/08/28 12:27:06 INFO Configuration.deprecation: session.id is deprecated. Instead, use dfs.metrics.session-id</span>
<span style="color: #888888;">16/08/28 12:27:06 INFO jvm.JvmMetrics: Initializing JVM Metrics with processName=JobTracker, sessionId=</span>
<span style="color: #888888;">16/08/28 12:27:06 INFO input.FileInputFormat: Total input paths to process : 29</span>
<span style="color: #888888;">16/08/28 12:27:06 INFO mapreduce.JobSubmitter: number of splits:29</span>
<span style="color: #888888;">16/08/28 12:27:06 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_local1735263799_0001</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapreduce.Job: The url to track the job: http://localhost:8080/</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapreduce.Job: Running job: job_local1735263799_0001</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.LocalJobRunner: OutputCommitter set in config null</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO output.FileOutputCommitter: File Output Committer Algorithm version is 1</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.LocalJobRunner: OutputCommitter is org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.LocalJobRunner: Waiting for map tasks</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.LocalJobRunner: Starting task: attempt_local1735263799_0001_m_000000_0</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO output.FileOutputCommitter: File Output Committer Algorithm version is 1</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.Task: Using ResourceCalculatorProcessTree : [ ]</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.MapTask: Processing split: hdfs://localhost:9000/user/hadoop/input/log4j.properties:0+11237</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.MapTask: (EQUATOR) 0 kvi 26214396(104857584)</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.MapTask: mapreduce.task.io.sort.mb: 100</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.MapTask: soft limit at 83886080</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.MapTask: bufstart = 0; bufvoid = 104857600</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.MapTask: kvstart = 26214396; length = 6553600</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.MapTask: Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.LocalJobRunner: </span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.MapTask: Starting flush of map output</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.MapTask: Spilling map output</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.MapTask: bufstart = 0; bufend = 279; bufvoid = 104857600</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.MapTask: kvstart = 26214396(104857584); kvend = 26214364(104857456); length = 33/6553600</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.MapTask: Finished spill 0</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.Task: Task:attempt_local1735263799_0001_m_000000_0 is done. And is in the process of committing</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.LocalJobRunner: map</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.Task: Task 'attempt_local1735263799_0001_m_000000_0' done.</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.LocalJobRunner: Finishing task: attempt_local1735263799_0001_m_000000_0</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.LocalJobRunner: Starting task: attempt_local1735263799_0001_m_000001_0</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO output.FileOutputCommitter: File Output Committer Algorithm version is 1</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.Task: Using ResourceCalculatorProcessTree : [ ]</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.MapTask: Processing split: hdfs://localhost:9000/user/hadoop/input/hadoop-policy.xml:0+9683</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.MapTask: (EQUATOR) 0 kvi 26214396(104857584)</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.MapTask: mapreduce.task.io.sort.mb: 100</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.MapTask: soft limit at 83886080</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.MapTask: bufstart = 0; bufvoid = 104857600</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.MapTask: kvstart = 26214396; length = 6553600</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.MapTask: Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.LocalJobRunner: </span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.MapTask: Starting flush of map output</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.MapTask: Spilling map output</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.MapTask: bufstart = 0; bufend = 17; bufvoid = 104857600</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.MapTask: kvstart = 26214396(104857584); kvend = 26214396(104857584); length = 1/6553600</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.MapTask: Finished spill 0</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.Task: Task:attempt_local1735263799_0001_m_000001_0 is done. And is in the process of committing</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.LocalJobRunner: map</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.Task: Task 'attempt_local1735263799_0001_m_000001_0' done.</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.LocalJobRunner: Finishing task: attempt_local1735263799_0001_m_000001_0</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.LocalJobRunner: Starting task: attempt_local1735263799_0001_m_000002_0</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO output.FileOutputCommitter: File Output Committer Algorithm version is 1</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.Task: Using ResourceCalculatorProcessTree : [ ]</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.MapTask: Processing split: hdfs://localhost:9000/user/hadoop/input/kms-site.xml:0+5511</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.MapTask: (EQUATOR) 0 kvi 26214396(104857584)</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.MapTask: mapreduce.task.io.sort.mb: 100</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.MapTask: soft limit at 83886080</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.MapTask: bufstart = 0; bufvoid = 104857600</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.MapTask: kvstart = 26214396; length = 6553600</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.MapTask: Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.LocalJobRunner: </span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.MapTask: Starting flush of map output</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.Task: Task:attempt_local1735263799_0001_m_000002_0 is done. And is in the process of committing</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.LocalJobRunner: map</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.Task: Task 'attempt_local1735263799_0001_m_000002_0' done.</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.LocalJobRunner: Finishing task: attempt_local1735263799_0001_m_000002_0</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.LocalJobRunner: Starting task: attempt_local1735263799_0001_m_000003_0</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO output.FileOutputCommitter: File Output Committer Algorithm version is 1</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.Task: Using ResourceCalculatorProcessTree : [ ]</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.MapTask: Processing split: hdfs://localhost:9000/user/hadoop/input/yarn-env.sh:0+4567</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.MapTask: (EQUATOR) 0 kvi 26214396(104857584)</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.MapTask: mapreduce.task.io.sort.mb: 100</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.MapTask: soft limit at 83886080</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.MapTask: bufstart = 0; bufvoid = 104857600</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.MapTask: kvstart = 26214396; length = 6553600</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.MapTask: Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.LocalJobRunner: </span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.MapTask: Starting flush of map output</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.Task: Task:attempt_local1735263799_0001_m_000003_0 is done. And is in the process of committing</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.LocalJobRunner: map</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.Task: Task 'attempt_local1735263799_0001_m_000003_0' done.</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.LocalJobRunner: Finishing task: attempt_local1735263799_0001_m_000003_0</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.LocalJobRunner: Starting task: attempt_local1735263799_0001_m_000004_0</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO output.FileOutputCommitter: File Output Committer Algorithm version is 1</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.Task: Using ResourceCalculatorProcessTree : [ ]</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.MapTask: Processing split: hdfs://localhost:9000/user/hadoop/input/capacity-scheduler.xml:0+4436</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.MapTask: (EQUATOR) 0 kvi 26214396(104857584)</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.MapTask: mapreduce.task.io.sort.mb: 100</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.MapTask: soft limit at 83886080</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.MapTask: bufstart = 0; bufvoid = 104857600</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.MapTask: kvstart = 26214396; length = 6553600</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.MapTask: Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.LocalJobRunner: </span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.MapTask: Starting flush of map output</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.Task: Task:attempt_local1735263799_0001_m_000004_0 is done. And is in the process of committing</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.LocalJobRunner: map</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.Task: Task 'attempt_local1735263799_0001_m_000004_0' done.</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.LocalJobRunner: Finishing task: attempt_local1735263799_0001_m_000004_0</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.LocalJobRunner: Starting task: attempt_local1735263799_0001_m_000005_0</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO output.FileOutputCommitter: File Output Committer Algorithm version is 1</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.Task: Using ResourceCalculatorProcessTree : [ ]</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.MapTask: Processing split: hdfs://localhost:9000/user/hadoop/input/hadoop-env.sh:0+4269</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.MapTask: (EQUATOR) 0 kvi 26214396(104857584)</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.MapTask: mapreduce.task.io.sort.mb: 100</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.MapTask: soft limit at 83886080</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.MapTask: bufstart = 0; bufvoid = 104857600</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.MapTask: kvstart = 26214396; length = 6553600</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.MapTask: Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.LocalJobRunner: </span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.MapTask: Starting flush of map output</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.MapTask: Spilling map output</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.MapTask: bufstart = 0; bufend = 50; bufvoid = 104857600</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.MapTask: kvstart = 26214396(104857584); kvend = 26214392(104857568); length = 5/6553600</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.MapTask: Finished spill 0</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.Task: Task:attempt_local1735263799_0001_m_000005_0 is done. And is in the process of committing</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.LocalJobRunner: map</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.Task: Task 'attempt_local1735263799_0001_m_000005_0' done.</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.LocalJobRunner: Finishing task: attempt_local1735263799_0001_m_000005_0</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.LocalJobRunner: Starting task: attempt_local1735263799_0001_m_000006_0</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO output.FileOutputCommitter: File Output Committer Algorithm version is 1</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.Task: Using ResourceCalculatorProcessTree : [ ]</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.MapTask: Processing split: hdfs://localhost:9000/user/hadoop/input/mapred-queues.xml.template:0+4113</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.MapTask: (EQUATOR) 0 kvi 26214396(104857584)</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.MapTask: mapreduce.task.io.sort.mb: 100</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.MapTask: soft limit at 83886080</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.MapTask: bufstart = 0; bufvoid = 104857600</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.MapTask: kvstart = 26214396; length = 6553600</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.MapTask: Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.LocalJobRunner: </span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.MapTask: Starting flush of map output</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.Task: Task:attempt_local1735263799_0001_m_000006_0 is done. And is in the process of committing</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.LocalJobRunner: map</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.Task: Task 'attempt_local1735263799_0001_m_000006_0' done.</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.LocalJobRunner: Finishing task: attempt_local1735263799_0001_m_000006_0</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.LocalJobRunner: Starting task: attempt_local1735263799_0001_m_000007_0</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO output.FileOutputCommitter: File Output Committer Algorithm version is 1</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.Task: Using ResourceCalculatorProcessTree : [ ]</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.MapTask: Processing split: hdfs://localhost:9000/user/hadoop/input/hadoop-env.cmd:0+3589</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.MapTask: (EQUATOR) 0 kvi 26214396(104857584)</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.MapTask: mapreduce.task.io.sort.mb: 100</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.MapTask: soft limit at 83886080</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.MapTask: bufstart = 0; bufvoid = 104857600</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.MapTask: kvstart = 26214396; length = 6553600</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.MapTask: Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.LocalJobRunner: </span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.MapTask: Starting flush of map output</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.MapTask: Spilling map output</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.MapTask: bufstart = 0; bufend = 50; bufvoid = 104857600</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.MapTask: kvstart = 26214396(104857584); kvend = 26214392(104857568); length = 5/6553600</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.MapTask: Finished spill 0</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.Task: Task:attempt_local1735263799_0001_m_000007_0 is done. And is in the process of committing</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.LocalJobRunner: map</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.Task: Task 'attempt_local1735263799_0001_m_000007_0' done.</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.LocalJobRunner: Finishing task: attempt_local1735263799_0001_m_000007_0</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.LocalJobRunner: Starting task: attempt_local1735263799_0001_m_000008_0</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO output.FileOutputCommitter: File Output Committer Algorithm version is 1</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.Task: Using ResourceCalculatorProcessTree : [ ]</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.MapTask: Processing split: hdfs://localhost:9000/user/hadoop/input/kms-acls.xml:0+3518</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.MapTask: (EQUATOR) 0 kvi 26214396(104857584)</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.MapTask: mapreduce.task.io.sort.mb: 100</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.MapTask: soft limit at 83886080</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.MapTask: bufstart = 0; bufvoid = 104857600</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.MapTask: kvstart = 26214396; length = 6553600</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.MapTask: Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.LocalJobRunner: </span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.MapTask: Starting flush of map output</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.Task: Task:attempt_local1735263799_0001_m_000008_0 is done. And is in the process of committing</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.LocalJobRunner: map</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.Task: Task 'attempt_local1735263799_0001_m_000008_0' done.</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.LocalJobRunner: Finishing task: attempt_local1735263799_0001_m_000008_0</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.LocalJobRunner: Starting task: attempt_local1735263799_0001_m_000009_0</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO output.FileOutputCommitter: File Output Committer Algorithm version is 1</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.Task: Using ResourceCalculatorProcessTree : [ ]</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.MapTask: Processing split: hdfs://localhost:9000/user/hadoop/input/hadoop-metrics2.properties:0+2598</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.MapTask: (EQUATOR) 0 kvi 26214396(104857584)</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.MapTask: mapreduce.task.io.sort.mb: 100</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.MapTask: soft limit at 83886080</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.MapTask: bufstart = 0; bufvoid = 104857600</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.MapTask: kvstart = 26214396; length = 6553600</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.MapTask: Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.LocalJobRunner: </span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.MapTask: Starting flush of map output</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.Task: Task:attempt_local1735263799_0001_m_000009_0 is done. And is in the process of committing</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.LocalJobRunner: map</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.Task: Task 'attempt_local1735263799_0001_m_000009_0' done.</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.LocalJobRunner: Finishing task: attempt_local1735263799_0001_m_000009_0</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.LocalJobRunner: Starting task: attempt_local1735263799_0001_m_000010_0</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO output.FileOutputCommitter: File Output Committer Algorithm version is 1</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.Task: Using ResourceCalculatorProcessTree : [ ]</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.MapTask: Processing split: hdfs://localhost:9000/user/hadoop/input/hadoop-metrics.properties:0+2490</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.MapTask: (EQUATOR) 0 kvi 26214396(104857584)</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.MapTask: mapreduce.task.io.sort.mb: 100</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.MapTask: soft limit at 83886080</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.MapTask: bufstart = 0; bufvoid = 104857600</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.MapTask: kvstart = 26214396; length = 6553600</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.MapTask: Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.LocalJobRunner: </span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.MapTask: Starting flush of map output</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.MapTask: Spilling map output</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.MapTask: bufstart = 0; bufend = 170; bufvoid = 104857600</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.MapTask: kvstart = 26214396(104857584); kvend = 26214364(104857456); length = 33/6553600</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.MapTask: Finished spill 0</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.Task: Task:attempt_local1735263799_0001_m_000010_0 is done. And is in the process of committing</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.LocalJobRunner: map</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.Task: Task 'attempt_local1735263799_0001_m_000010_0' done.</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.LocalJobRunner: Finishing task: attempt_local1735263799_0001_m_000010_0</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.LocalJobRunner: Starting task: attempt_local1735263799_0001_m_000011_0</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO output.FileOutputCommitter: File Output Committer Algorithm version is 1</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.Task: Using ResourceCalculatorProcessTree : [ ]</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.MapTask: Processing split: hdfs://localhost:9000/user/hadoop/input/ssl-client.xml.example:0+2316</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.MapTask: (EQUATOR) 0 kvi 26214396(104857584)</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.MapTask: mapreduce.task.io.sort.mb: 100</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.MapTask: soft limit at 83886080</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.MapTask: bufstart = 0; bufvoid = 104857600</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.MapTask: kvstart = 26214396; length = 6553600</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.MapTask: Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.LocalJobRunner: </span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.MapTask: Starting flush of map output</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.Task: Task:attempt_local1735263799_0001_m_000011_0 is done. And is in the process of committing</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.LocalJobRunner: map</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.Task: Task 'attempt_local1735263799_0001_m_000011_0' done.</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.LocalJobRunner: Finishing task: attempt_local1735263799_0001_m_000011_0</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.LocalJobRunner: Starting task: attempt_local1735263799_0001_m_000012_0</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO output.FileOutputCommitter: File Output Committer Algorithm version is 1</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.Task: Using ResourceCalculatorProcessTree : [ ]</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.MapTask: Processing split: hdfs://localhost:9000/user/hadoop/input/ssl-server.xml.example:0+2268</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.MapTask: (EQUATOR) 0 kvi 26214396(104857584)</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.MapTask: mapreduce.task.io.sort.mb: 100</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.MapTask: soft limit at 83886080</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.MapTask: bufstart = 0; bufvoid = 104857600</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.MapTask: kvstart = 26214396; length = 6553600</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.MapTask: Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.LocalJobRunner: </span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.MapTask: Starting flush of map output</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.Task: Task:attempt_local1735263799_0001_m_000012_0 is done. And is in the process of committing</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.LocalJobRunner: map</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.Task: Task 'attempt_local1735263799_0001_m_000012_0' done.</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.LocalJobRunner: Finishing task: attempt_local1735263799_0001_m_000012_0</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.LocalJobRunner: Starting task: attempt_local1735263799_0001_m_000013_0</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO output.FileOutputCommitter: File Output Committer Algorithm version is 1</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.Task: Using ResourceCalculatorProcessTree : [ ]</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.MapTask: Processing split: hdfs://localhost:9000/user/hadoop/input/yarn-env.cmd:0+2191</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.MapTask: (EQUATOR) 0 kvi 26214396(104857584)</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.MapTask: mapreduce.task.io.sort.mb: 100</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.MapTask: soft limit at 83886080</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.MapTask: bufstart = 0; bufvoid = 104857600</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.MapTask: kvstart = 26214396; length = 6553600</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.MapTask: Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.LocalJobRunner: </span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.MapTask: Starting flush of map output</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.Task: Task:attempt_local1735263799_0001_m_000013_0 is done. And is in the process of committing</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.LocalJobRunner: map</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.Task: Task 'attempt_local1735263799_0001_m_000013_0' done.</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.LocalJobRunner: Finishing task: attempt_local1735263799_0001_m_000013_0</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.LocalJobRunner: Starting task: attempt_local1735263799_0001_m_000014_0</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO output.FileOutputCommitter: File Output Committer Algorithm version is 1</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.Task: Using ResourceCalculatorProcessTree : [ ]</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.MapTask: Processing split: hdfs://localhost:9000/user/hadoop/input/httpfs-log4j.properties:0+1657</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.MapTask: (EQUATOR) 0 kvi 26214396(104857584)</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.MapTask: mapreduce.task.io.sort.mb: 100</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.MapTask: soft limit at 83886080</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.MapTask: bufstart = 0; bufvoid = 104857600</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.MapTask: kvstart = 26214396; length = 6553600</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.MapTask: Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.LocalJobRunner: </span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.MapTask: Starting flush of map output</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.Task: Task:attempt_local1735263799_0001_m_000014_0 is done. And is in the process of committing</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.LocalJobRunner: map</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.Task: Task 'attempt_local1735263799_0001_m_000014_0' done.</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.LocalJobRunner: Finishing task: attempt_local1735263799_0001_m_000014_0</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.LocalJobRunner: Starting task: attempt_local1735263799_0001_m_000015_0</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO output.FileOutputCommitter: File Output Committer Algorithm version is 1</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.Task: Using ResourceCalculatorProcessTree : [ ]</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.MapTask: Processing split: hdfs://localhost:9000/user/hadoop/input/kms-log4j.properties:0+1631</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.MapTask: (EQUATOR) 0 kvi 26214396(104857584)</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.MapTask: mapreduce.task.io.sort.mb: 100</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.MapTask: soft limit at 83886080</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.MapTask: bufstart = 0; bufvoid = 104857600</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.MapTask: kvstart = 26214396; length = 6553600</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.MapTask: Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.LocalJobRunner: </span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.MapTask: Starting flush of map output</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.Task: Task:attempt_local1735263799_0001_m_000015_0 is done. And is in the process of committing</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.LocalJobRunner: map</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.Task: Task 'attempt_local1735263799_0001_m_000015_0' done.</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.LocalJobRunner: Finishing task: attempt_local1735263799_0001_m_000015_0</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.LocalJobRunner: Starting task: attempt_local1735263799_0001_m_000016_0</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO output.FileOutputCommitter: File Output Committer Algorithm version is 1</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.Task: Using ResourceCalculatorProcessTree : [ ]</span>
<span style="color: #888888;">16/08/28 12:27:07 INFO mapred.MapTask: Processing split: hdfs://localhost:9000/user/hadoop/input/kms-env.sh:0+1527</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO mapred.MapTask: (EQUATOR) 0 kvi 26214396(104857584)</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO mapred.MapTask: mapreduce.task.io.sort.mb: 100</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO mapred.MapTask: soft limit at 83886080</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO mapred.MapTask: bufstart = 0; bufvoid = 104857600</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO mapred.MapTask: kvstart = 26214396; length = 6553600</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO mapred.MapTask: Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO mapred.LocalJobRunner: </span>
<span style="color: #888888;">16/08/28 12:27:08 INFO mapred.MapTask: Starting flush of map output</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO mapred.Task: Task:attempt_local1735263799_0001_m_000016_0 is done. And is in the process of committing</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO mapred.LocalJobRunner: map</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO mapred.Task: Task 'attempt_local1735263799_0001_m_000016_0' done.</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO mapred.LocalJobRunner: Finishing task: attempt_local1735263799_0001_m_000016_0</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO mapred.LocalJobRunner: Starting task: attempt_local1735263799_0001_m_000017_0</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO output.FileOutputCommitter: File Output Committer Algorithm version is 1</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO mapred.Task: Using ResourceCalculatorProcessTree : [ ]</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO mapred.MapTask: Processing split: hdfs://localhost:9000/user/hadoop/input/httpfs-env.sh:0+1449</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO mapreduce.Job: Job job_local1735263799_0001 running in uber mode : false</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO mapreduce.Job: map 100% reduce 0%</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO mapred.MapTask: (EQUATOR) 0 kvi 26214396(104857584)</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO mapred.MapTask: mapreduce.task.io.sort.mb: 100</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO mapred.MapTask: soft limit at 83886080</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO mapred.MapTask: bufstart = 0; bufvoid = 104857600</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO mapred.MapTask: kvstart = 26214396; length = 6553600</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO mapred.MapTask: Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO mapred.LocalJobRunner: </span>
<span style="color: #888888;">16/08/28 12:27:08 INFO mapred.MapTask: Starting flush of map output</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO mapred.Task: Task:attempt_local1735263799_0001_m_000017_0 is done. And is in the process of committing</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO mapred.LocalJobRunner: map</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO mapred.Task: Task 'attempt_local1735263799_0001_m_000017_0' done.</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO mapred.LocalJobRunner: Finishing task: attempt_local1735263799_0001_m_000017_0</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO mapred.LocalJobRunner: Starting task: attempt_local1735263799_0001_m_000018_0</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO output.FileOutputCommitter: File Output Committer Algorithm version is 1</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO mapred.Task: Using ResourceCalculatorProcessTree : [ ]</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO mapred.MapTask: Processing split: hdfs://localhost:9000/user/hadoop/input/mapred-env.sh:0+1383</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO mapred.MapTask: (EQUATOR) 0 kvi 26214396(104857584)</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO mapred.MapTask: mapreduce.task.io.sort.mb: 100</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO mapred.MapTask: soft limit at 83886080</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO mapred.MapTask: bufstart = 0; bufvoid = 104857600</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO mapred.MapTask: kvstart = 26214396; length = 6553600</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO mapred.MapTask: Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO mapred.LocalJobRunner: </span>
<span style="color: #888888;">16/08/28 12:27:08 INFO mapred.MapTask: Starting flush of map output</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO mapred.Task: Task:attempt_local1735263799_0001_m_000018_0 is done. And is in the process of committing</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO mapred.LocalJobRunner: map</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO mapred.Task: Task 'attempt_local1735263799_0001_m_000018_0' done.</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO mapred.LocalJobRunner: Finishing task: attempt_local1735263799_0001_m_000018_0</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO mapred.LocalJobRunner: Starting task: attempt_local1735263799_0001_m_000019_0</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO output.FileOutputCommitter: File Output Committer Algorithm version is 1</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO mapred.Task: Using ResourceCalculatorProcessTree : [ ]</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO mapred.MapTask: Processing split: hdfs://localhost:9000/user/hadoop/input/configuration.xsl:0+1335</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO mapred.MapTask: (EQUATOR) 0 kvi 26214396(104857584)</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO mapred.MapTask: mapreduce.task.io.sort.mb: 100</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO mapred.MapTask: soft limit at 83886080</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO mapred.MapTask: bufstart = 0; bufvoid = 104857600</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO mapred.MapTask: kvstart = 26214396; length = 6553600</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO mapred.MapTask: Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO mapred.LocalJobRunner: </span>
<span style="color: #888888;">16/08/28 12:27:08 INFO mapred.MapTask: Starting flush of map output</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO mapred.Task: Task:attempt_local1735263799_0001_m_000019_0 is done. And is in the process of committing</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO mapred.LocalJobRunner: map</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO mapred.Task: Task 'attempt_local1735263799_0001_m_000019_0' done.</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO mapred.LocalJobRunner: Finishing task: attempt_local1735263799_0001_m_000019_0</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO mapred.LocalJobRunner: Starting task: attempt_local1735263799_0001_m_000020_0</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO output.FileOutputCommitter: File Output Committer Algorithm version is 1</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO mapred.Task: Using ResourceCalculatorProcessTree : [ ]</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO mapred.MapTask: Processing split: hdfs://localhost:9000/user/hadoop/input/mapred-env.cmd:0+931</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO mapred.MapTask: (EQUATOR) 0 kvi 26214396(104857584)</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO mapred.MapTask: mapreduce.task.io.sort.mb: 100</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO mapred.MapTask: soft limit at 83886080</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO mapred.MapTask: bufstart = 0; bufvoid = 104857600</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO mapred.MapTask: kvstart = 26214396; length = 6553600</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO mapred.MapTask: Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO mapred.LocalJobRunner: </span>
<span style="color: #888888;">16/08/28 12:27:08 INFO mapred.MapTask: Starting flush of map output</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO mapred.Task: Task:attempt_local1735263799_0001_m_000020_0 is done. And is in the process of committing</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO mapred.LocalJobRunner: map</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO mapred.Task: Task 'attempt_local1735263799_0001_m_000020_0' done.</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO mapred.LocalJobRunner: Finishing task: attempt_local1735263799_0001_m_000020_0</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO mapred.LocalJobRunner: Starting task: attempt_local1735263799_0001_m_000021_0</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO output.FileOutputCommitter: File Output Committer Algorithm version is 1</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO mapred.Task: Using ResourceCalculatorProcessTree : [ ]</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO mapred.MapTask: Processing split: hdfs://localhost:9000/user/hadoop/input/core-site.xml:0+880</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO mapred.MapTask: (EQUATOR) 0 kvi 26214396(104857584)</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO mapred.MapTask: mapreduce.task.io.sort.mb: 100</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO mapred.MapTask: soft limit at 83886080</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO mapred.MapTask: bufstart = 0; bufvoid = 104857600</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO mapred.MapTask: kvstart = 26214396; length = 6553600</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO mapred.MapTask: Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO mapred.LocalJobRunner: </span>
<span style="color: #888888;">16/08/28 12:27:08 INFO mapred.MapTask: Starting flush of map output</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO mapred.Task: Task:attempt_local1735263799_0001_m_000021_0 is done. And is in the process of committing</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO mapred.LocalJobRunner: map</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO mapred.Task: Task 'attempt_local1735263799_0001_m_000021_0' done.</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO mapred.LocalJobRunner: Finishing task: attempt_local1735263799_0001_m_000021_0</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO mapred.LocalJobRunner: Starting task: attempt_local1735263799_0001_m_000022_0</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO output.FileOutputCommitter: File Output Committer Algorithm version is 1</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO mapred.Task: Using ResourceCalculatorProcessTree : [ ]</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO mapred.MapTask: Processing split: hdfs://localhost:9000/user/hadoop/input/hdfs-site.xml:0+863</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO mapred.MapTask: (EQUATOR) 0 kvi 26214396(104857584)</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO mapred.MapTask: mapreduce.task.io.sort.mb: 100</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO mapred.MapTask: soft limit at 83886080</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO mapred.MapTask: bufstart = 0; bufvoid = 104857600</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO mapred.MapTask: kvstart = 26214396; length = 6553600</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO mapred.MapTask: Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO mapred.LocalJobRunner: </span>
<span style="color: #888888;">16/08/28 12:27:08 INFO mapred.MapTask: Starting flush of map output</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO mapred.MapTask: Spilling map output</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO mapred.MapTask: bufstart = 0; bufend = 24; bufvoid = 104857600</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO mapred.MapTask: kvstart = 26214396(104857584); kvend = 26214396(104857584); length = 1/6553600</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO mapred.MapTask: Finished spill 0</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO mapred.Task: Task:attempt_local1735263799_0001_m_000022_0 is done. And is in the process of committing</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO mapred.LocalJobRunner: map</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO mapred.Task: Task 'attempt_local1735263799_0001_m_000022_0' done.</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO mapred.LocalJobRunner: Finishing task: attempt_local1735263799_0001_m_000022_0</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO mapred.LocalJobRunner: Starting task: attempt_local1735263799_0001_m_000023_0</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO output.FileOutputCommitter: File Output Committer Algorithm version is 1</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO mapred.Task: Using ResourceCalculatorProcessTree : [ ]</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO mapred.MapTask: Processing split: hdfs://localhost:9000/user/hadoop/input/mapred-site.xml.template:0+758</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO mapred.MapTask: (EQUATOR) 0 kvi 26214396(104857584)</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO mapred.MapTask: mapreduce.task.io.sort.mb: 100</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO mapred.MapTask: soft limit at 83886080</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO mapred.MapTask: bufstart = 0; bufvoid = 104857600</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO mapred.MapTask: kvstart = 26214396; length = 6553600</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO mapred.MapTask: Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO mapred.LocalJobRunner: </span>
<span style="color: #888888;">16/08/28 12:27:08 INFO mapred.MapTask: Starting flush of map output</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO mapred.Task: Task:attempt_local1735263799_0001_m_000023_0 is done. And is in the process of committing</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO mapred.LocalJobRunner: map</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO mapred.Task: Task 'attempt_local1735263799_0001_m_000023_0' done.</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO mapred.LocalJobRunner: Finishing task: attempt_local1735263799_0001_m_000023_0</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO mapred.LocalJobRunner: Starting task: attempt_local1735263799_0001_m_000024_0</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO output.FileOutputCommitter: File Output Committer Algorithm version is 1</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO mapred.Task: Using ResourceCalculatorProcessTree : [ ]</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO mapred.MapTask: Processing split: hdfs://localhost:9000/user/hadoop/input/yarn-site.xml:0+690</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO mapred.MapTask: (EQUATOR) 0 kvi 26214396(104857584)</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO mapred.MapTask: mapreduce.task.io.sort.mb: 100</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO mapred.MapTask: soft limit at 83886080</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO mapred.MapTask: bufstart = 0; bufvoid = 104857600</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO mapred.MapTask: kvstart = 26214396; length = 6553600</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO mapred.MapTask: Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO mapred.LocalJobRunner: </span>
<span style="color: #888888;">16/08/28 12:27:08 INFO mapred.MapTask: Starting flush of map output</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO mapred.Task: Task:attempt_local1735263799_0001_m_000024_0 is done. And is in the process of committing</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO mapred.LocalJobRunner: map</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO mapred.Task: Task 'attempt_local1735263799_0001_m_000024_0' done.</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO mapred.LocalJobRunner: Finishing task: attempt_local1735263799_0001_m_000024_0</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO mapred.LocalJobRunner: Starting task: attempt_local1735263799_0001_m_000025_0</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO output.FileOutputCommitter: File Output Committer Algorithm version is 1</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO mapred.Task: Using ResourceCalculatorProcessTree : [ ]</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO mapred.MapTask: Processing split: hdfs://localhost:9000/user/hadoop/input/httpfs-site.xml:0+620</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO mapred.MapTask: (EQUATOR) 0 kvi 26214396(104857584)</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO mapred.MapTask: mapreduce.task.io.sort.mb: 100</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO mapred.MapTask: soft limit at 83886080</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO mapred.MapTask: bufstart = 0; bufvoid = 104857600</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO mapred.MapTask: kvstart = 26214396; length = 6553600</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO mapred.MapTask: Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO mapred.LocalJobRunner: </span>
<span style="color: #888888;">16/08/28 12:27:08 INFO mapred.MapTask: Starting flush of map output</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO mapred.Task: Task:attempt_local1735263799_0001_m_000025_0 is done. And is in the process of committing</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO mapred.LocalJobRunner: map</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO mapred.Task: Task 'attempt_local1735263799_0001_m_000025_0' done.</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO mapred.LocalJobRunner: Finishing task: attempt_local1735263799_0001_m_000025_0</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO mapred.LocalJobRunner: Starting task: attempt_local1735263799_0001_m_000026_0</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO output.FileOutputCommitter: File Output Committer Algorithm version is 1</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO mapred.Task: Using ResourceCalculatorProcessTree : [ ]</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO mapred.MapTask: Processing split: hdfs://localhost:9000/user/hadoop/input/container-executor.cfg:0+318</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO mapred.MapTask: (EQUATOR) 0 kvi 26214396(104857584)</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO mapred.MapTask: mapreduce.task.io.sort.mb: 100</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO mapred.MapTask: soft limit at 83886080</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO mapred.MapTask: bufstart = 0; bufvoid = 104857600</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO mapred.MapTask: kvstart = 26214396; length = 6553600</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO mapred.MapTask: Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO mapred.LocalJobRunner: </span>
<span style="color: #888888;">16/08/28 12:27:08 INFO mapred.MapTask: Starting flush of map output</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO mapred.Task: Task:attempt_local1735263799_0001_m_000026_0 is done. And is in the process of committing</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO mapred.LocalJobRunner: map</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO mapred.Task: Task 'attempt_local1735263799_0001_m_000026_0' done.</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO mapred.LocalJobRunner: Finishing task: attempt_local1735263799_0001_m_000026_0</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO mapred.LocalJobRunner: Starting task: attempt_local1735263799_0001_m_000027_0</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO output.FileOutputCommitter: File Output Committer Algorithm version is 1</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO mapred.Task: Using ResourceCalculatorProcessTree : [ ]</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO mapred.MapTask: Processing split: hdfs://localhost:9000/user/hadoop/input/httpfs-signature.secret:0+21</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO mapred.MapTask: (EQUATOR) 0 kvi 26214396(104857584)</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO mapred.MapTask: mapreduce.task.io.sort.mb: 100</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO mapred.MapTask: soft limit at 83886080</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO mapred.MapTask: bufstart = 0; bufvoid = 104857600</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO mapred.MapTask: kvstart = 26214396; length = 6553600</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO mapred.MapTask: Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO mapred.LocalJobRunner: </span>
<span style="color: #888888;">16/08/28 12:27:08 INFO mapred.MapTask: Starting flush of map output</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO mapred.Task: Task:attempt_local1735263799_0001_m_000027_0 is done. And is in the process of committing</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO mapred.LocalJobRunner: map</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO mapred.Task: Task 'attempt_local1735263799_0001_m_000027_0' done.</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO mapred.LocalJobRunner: Finishing task: attempt_local1735263799_0001_m_000027_0</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO mapred.LocalJobRunner: Starting task: attempt_local1735263799_0001_m_000028_0</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO output.FileOutputCommitter: File Output Committer Algorithm version is 1</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO mapred.Task: Using ResourceCalculatorProcessTree : [ ]</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO mapred.MapTask: Processing split: hdfs://localhost:9000/user/hadoop/input/slaves:0+10</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO mapred.MapTask: (EQUATOR) 0 kvi 26214396(104857584)</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO mapred.MapTask: mapreduce.task.io.sort.mb: 100</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO mapred.MapTask: soft limit at 83886080</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO mapred.MapTask: bufstart = 0; bufvoid = 104857600</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO mapred.MapTask: kvstart = 26214396; length = 6553600</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO mapred.MapTask: Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO mapred.LocalJobRunner: </span>
<span style="color: #888888;">16/08/28 12:27:08 INFO mapred.MapTask: Starting flush of map output</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO mapred.Task: Task:attempt_local1735263799_0001_m_000028_0 is done. And is in the process of committing</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO mapred.LocalJobRunner: map</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO mapred.Task: Task 'attempt_local1735263799_0001_m_000028_0' done.</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO mapred.LocalJobRunner: Finishing task: attempt_local1735263799_0001_m_000028_0</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO mapred.LocalJobRunner: map task executor complete.</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO mapred.LocalJobRunner: Waiting for reduce tasks</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO mapred.LocalJobRunner: Starting task: attempt_local1735263799_0001_r_000000_0</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO output.FileOutputCommitter: File Output Committer Algorithm version is 1</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO mapred.Task: Using ResourceCalculatorProcessTree : [ ]</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO mapred.ReduceTask: Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@3be15b04</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO reduce.MergeManagerImpl: MergerManager: memoryLimit=371038624, maxSingleShuffleLimit=92759656, mergeThreshold=244885504, ioSortFactor=10, memToMemMergeOutputsThreshold=10</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO reduce.EventFetcher: attempt_local1735263799_0001_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO reduce.LocalFetcher: localfetcher#1 about to shuffle output of map attempt_local1735263799_0001_m_000016_0 decomp: 2 len: 6 to MEMORY</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO reduce.InMemoryMapOutput: Read 2 bytes from map-output for attempt_local1735263799_0001_m_000016_0</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 2, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->2</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO reduce.LocalFetcher: localfetcher#1 about to shuffle output of map attempt_local1735263799_0001_m_000003_0 decomp: 2 len: 6 to MEMORY</span>
<span style="color: #888888;">16/08/28 12:27:08 WARN io.ReadaheadPool: Failed readahead on ifile</span>
<span style="color: #888888;">EBADF: Bad file descriptor</span>
<span style="color: #888888;"> at org.apache.hadoop.io.nativeio.NativeIO$POSIX.posix_fadvise(Native Method)</span>
<span style="color: #888888;"> at org.apache.hadoop.io.nativeio.NativeIO$POSIX.posixFadviseIfPossible(NativeIO.java:267)</span>
<span style="color: #888888;"> at org.apache.hadoop.io.nativeio.NativeIO$POSIX$CacheManipulator.posixFadviseIfPossible(NativeIO.java:146)</span>
<span style="color: #888888;"> at org.apache.hadoop.io.ReadaheadPool$ReadaheadRequestImpl.run(ReadaheadPool.java:206)</span>
<span style="color: #888888;"> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)</span>
<span style="color: #888888;"> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)</span>
<span style="color: #888888;"> at java.lang.Thread.run(Thread.java:745)</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO reduce.InMemoryMapOutput: Read 2 bytes from map-output for attempt_local1735263799_0001_m_000003_0</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 2, inMemoryMapOutputs.size() -> 2, commitMemory -> 2, usedMemory ->4</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO reduce.LocalFetcher: localfetcher#1 about to shuffle output of map attempt_local1735263799_0001_m_000004_0 decomp: 2 len: 6 to MEMORY</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO reduce.InMemoryMapOutput: Read 2 bytes from map-output for attempt_local1735263799_0001_m_000004_0</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 2, inMemoryMapOutputs.size() -> 3, commitMemory -> 4, usedMemory ->6</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO reduce.LocalFetcher: localfetcher#1 about to shuffle output of map attempt_local1735263799_0001_m_000017_0 decomp: 2 len: 6 to MEMORY</span>
<span style="color: #888888;">16/08/28 12:27:08 WARN io.ReadaheadPool: Failed readahead on ifile</span>
<span style="color: #888888;">EBADF: Bad file descriptor</span>
<span style="color: #888888;"> at org.apache.hadoop.io.nativeio.NativeIO$POSIX.posix_fadvise(Native Method)</span>
<span style="color: #888888;"> at org.apache.hadoop.io.nativeio.NativeIO$POSIX.posixFadviseIfPossible(NativeIO.java:267)</span>
<span style="color: #888888;"> at org.apache.hadoop.io.nativeio.NativeIO$POSIX$CacheManipulator.posixFadviseIfPossible(NativeIO.java:146)</span>
<span style="color: #888888;"> at org.apache.hadoop.io.ReadaheadPool$ReadaheadRequestImpl.run(ReadaheadPool.java:206)</span>
<span style="color: #888888;"> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)</span>
<span style="color: #888888;"> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)</span>
<span style="color: #888888;"> at java.lang.Thread.run(Thread.java:745)</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO reduce.InMemoryMapOutput: Read 2 bytes from map-output for attempt_local1735263799_0001_m_000017_0</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 2, inMemoryMapOutputs.size() -> 4, commitMemory -> 6, usedMemory ->8</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO reduce.LocalFetcher: localfetcher#1 about to shuffle output of map attempt_local1735263799_0001_m_000027_0 decomp: 2 len: 6 to MEMORY</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO reduce.InMemoryMapOutput: Read 2 bytes from map-output for attempt_local1735263799_0001_m_000027_0</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 2, inMemoryMapOutputs.size() -> 5, commitMemory -> 8, usedMemory ->10</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO reduce.LocalFetcher: localfetcher#1 about to shuffle output of map attempt_local1735263799_0001_m_000002_0 decomp: 2 len: 6 to MEMORY</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO reduce.InMemoryMapOutput: Read 2 bytes from map-output for attempt_local1735263799_0001_m_000002_0</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 2, inMemoryMapOutputs.size() -> 6, commitMemory -> 10, usedMemory ->12</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO reduce.LocalFetcher: localfetcher#1 about to shuffle output of map attempt_local1735263799_0001_m_000028_0 decomp: 2 len: 6 to MEMORY</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO reduce.InMemoryMapOutput: Read 2 bytes from map-output for attempt_local1735263799_0001_m_000028_0</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 2, inMemoryMapOutputs.size() -> 7, commitMemory -> 12, usedMemory ->14</span>
<span style="color: #888888;">16/08/28 12:27:08 WARN io.ReadaheadPool: Failed readahead on ifile</span>
<span style="color: #888888;">EBADF: Bad file descriptor</span>
<span style="color: #888888;"> at org.apache.hadoop.io.nativeio.NativeIO$POSIX.posix_fadvise(Native Method)</span>
<span style="color: #888888;"> at org.apache.hadoop.io.nativeio.NativeIO$POSIX.posixFadviseIfPossible(NativeIO.java:267)</span>
<span style="color: #888888;"> at org.apache.hadoop.io.nativeio.NativeIO$POSIX$CacheManipulator.posixFadviseIfPossible(NativeIO.java:146)</span>
<span style="color: #888888;"> at org.apache.hadoop.io.ReadaheadPool$ReadaheadRequestImpl.run(ReadaheadPool.java:206)</span>
<span style="color: #888888;"> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)</span>
<span style="color: #888888;"> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)</span>
<span style="color: #888888;"> at java.lang.Thread.run(Thread.java:745)</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO reduce.LocalFetcher: localfetcher#1 about to shuffle output of map attempt_local1735263799_0001_m_000015_0 decomp: 2 len: 6 to MEMORY</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO reduce.InMemoryMapOutput: Read 2 bytes from map-output for attempt_local1735263799_0001_m_000015_0</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 2, inMemoryMapOutputs.size() -> 8, commitMemory -> 14, usedMemory ->16</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO reduce.LocalFetcher: localfetcher#1 about to shuffle output of map attempt_local1735263799_0001_m_000019_0 decomp: 2 len: 6 to MEMORY</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO reduce.InMemoryMapOutput: Read 2 bytes from map-output for attempt_local1735263799_0001_m_000019_0</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 2, inMemoryMapOutputs.size() -> 9, commitMemory -> 16, usedMemory ->18</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO reduce.LocalFetcher: localfetcher#1 about to shuffle output of map attempt_local1735263799_0001_m_000020_0 decomp: 2 len: 6 to MEMORY</span>
<span style="color: #888888;">16/08/28 12:27:08 WARN io.ReadaheadPool: Failed readahead on ifile</span>
<span style="color: #888888;">EBADF: Bad file descriptor</span>
<span style="color: #888888;"> at org.apache.hadoop.io.nativeio.NativeIO$POSIX.posix_fadvise(Native Method)</span>
<span style="color: #888888;"> at org.apache.hadoop.io.nativeio.NativeIO$POSIX.posixFadviseIfPossible(NativeIO.java:267)</span>
<span style="color: #888888;"> at org.apache.hadoop.io.nativeio.NativeIO$POSIX$CacheManipulator.posixFadviseIfPossible(NativeIO.java:146)</span>
<span style="color: #888888;"> at org.apache.hadoop.io.ReadaheadPool$ReadaheadRequestImpl.run(ReadaheadPool.java:206)</span>
<span style="color: #888888;"> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)</span>
<span style="color: #888888;"> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)</span>
<span style="color: #888888;"> at java.lang.Thread.run(Thread.java:745)</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO reduce.InMemoryMapOutput: Read 2 bytes from map-output for attempt_local1735263799_0001_m_000020_0</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 2, inMemoryMapOutputs.size() -> 10, commitMemory -> 18, usedMemory ->20</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO reduce.LocalFetcher: localfetcher#1 about to shuffle output of map attempt_local1735263799_0001_m_000007_0 decomp: 29 len: 33 to MEMORY</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO reduce.InMemoryMapOutput: Read 29 bytes from map-output for attempt_local1735263799_0001_m_000007_0</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 29, inMemoryMapOutputs.size() -> 11, commitMemory -> 20, usedMemory ->49</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO reduce.LocalFetcher: localfetcher#1 about to shuffle output of map attempt_local1735263799_0001_m_000018_0 decomp: 2 len: 6 to MEMORY</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO reduce.InMemoryMapOutput: Read 2 bytes from map-output for attempt_local1735263799_0001_m_000018_0</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 2, inMemoryMapOutputs.size() -> 12, commitMemory -> 49, usedMemory ->51</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO reduce.LocalFetcher: localfetcher#1 about to shuffle output of map attempt_local1735263799_0001_m_000005_0 decomp: 29 len: 33 to MEMORY</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO reduce.InMemoryMapOutput: Read 29 bytes from map-output for attempt_local1735263799_0001_m_000005_0</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 29, inMemoryMapOutputs.size() -> 13, commitMemory -> 51, usedMemory ->80</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO reduce.LocalFetcher: localfetcher#1 about to shuffle output of map attempt_local1735263799_0001_m_000006_0 decomp: 2 len: 6 to MEMORY</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO reduce.InMemoryMapOutput: Read 2 bytes from map-output for attempt_local1735263799_0001_m_000006_0</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 2, inMemoryMapOutputs.size() -> 14, commitMemory -> 80, usedMemory ->82</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO reduce.LocalFetcher: localfetcher#1 about to shuffle output of map attempt_local1735263799_0001_m_000010_0 decomp: 109 len: 113 to MEMORY</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO reduce.InMemoryMapOutput: Read 109 bytes from map-output for attempt_local1735263799_0001_m_000010_0</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 109, inMemoryMapOutputs.size() -> 15, commitMemory -> 82, usedMemory ->191</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO reduce.LocalFetcher: localfetcher#1 about to shuffle output of map attempt_local1735263799_0001_m_000023_0 decomp: 2 len: 6 to MEMORY</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO reduce.InMemoryMapOutput: Read 2 bytes from map-output for attempt_local1735263799_0001_m_000023_0</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 2, inMemoryMapOutputs.size() -> 16, commitMemory -> 191, usedMemory ->193</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO reduce.LocalFetcher: localfetcher#1 about to shuffle output of map attempt_local1735263799_0001_m_000008_0 decomp: 2 len: 6 to MEMORY</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO reduce.InMemoryMapOutput: Read 2 bytes from map-output for attempt_local1735263799_0001_m_000008_0</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 2, inMemoryMapOutputs.size() -> 17, commitMemory -> 193, usedMemory ->195</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO reduce.LocalFetcher: localfetcher#1 about to shuffle output of map attempt_local1735263799_0001_m_000021_0 decomp: 2 len: 6 to MEMORY</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO reduce.InMemoryMapOutput: Read 2 bytes from map-output for attempt_local1735263799_0001_m_000021_0</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 2, inMemoryMapOutputs.size() -> 18, commitMemory -> 195, usedMemory ->197</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO reduce.LocalFetcher: localfetcher#1 about to shuffle output of map attempt_local1735263799_0001_m_000022_0 decomp: 28 len: 32 to MEMORY</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO reduce.InMemoryMapOutput: Read 28 bytes from map-output for attempt_local1735263799_0001_m_000022_0</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 28, inMemoryMapOutputs.size() -> 19, commitMemory -> 197, usedMemory ->225</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO reduce.LocalFetcher: localfetcher#1 about to shuffle output of map attempt_local1735263799_0001_m_000009_0 decomp: 2 len: 6 to MEMORY</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO reduce.InMemoryMapOutput: Read 2 bytes from map-output for attempt_local1735263799_0001_m_000009_0</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 2, inMemoryMapOutputs.size() -> 20, commitMemory -> 225, usedMemory ->227</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO reduce.LocalFetcher: localfetcher#1 about to shuffle output of map attempt_local1735263799_0001_m_000000_0 decomp: 135 len: 139 to MEMORY</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO reduce.InMemoryMapOutput: Read 135 bytes from map-output for attempt_local1735263799_0001_m_000000_0</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 135, inMemoryMapOutputs.size() -> 21, commitMemory -> 227, usedMemory ->362</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO reduce.LocalFetcher: localfetcher#1 about to shuffle output of map attempt_local1735263799_0001_m_000026_0 decomp: 2 len: 6 to MEMORY</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO reduce.InMemoryMapOutput: Read 2 bytes from map-output for attempt_local1735263799_0001_m_000026_0</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 2, inMemoryMapOutputs.size() -> 22, commitMemory -> 362, usedMemory ->364</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO reduce.LocalFetcher: localfetcher#1 about to shuffle output of map attempt_local1735263799_0001_m_000013_0 decomp: 2 len: 6 to MEMORY</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO reduce.InMemoryMapOutput: Read 2 bytes from map-output for attempt_local1735263799_0001_m_000013_0</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 2, inMemoryMapOutputs.size() -> 23, commitMemory -> 364, usedMemory ->366</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO reduce.LocalFetcher: localfetcher#1 about to shuffle output of map attempt_local1735263799_0001_m_000014_0 decomp: 2 len: 6 to MEMORY</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO reduce.InMemoryMapOutput: Read 2 bytes from map-output for attempt_local1735263799_0001_m_000014_0</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 2, inMemoryMapOutputs.size() -> 24, commitMemory -> 366, usedMemory ->368</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO reduce.LocalFetcher: localfetcher#1 about to shuffle output of map attempt_local1735263799_0001_m_000001_0 decomp: 21 len: 25 to MEMORY</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO reduce.InMemoryMapOutput: Read 21 bytes from map-output for attempt_local1735263799_0001_m_000001_0</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 21, inMemoryMapOutputs.size() -> 25, commitMemory -> 368, usedMemory ->389</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO reduce.LocalFetcher: localfetcher#1 about to shuffle output of map attempt_local1735263799_0001_m_000024_0 decomp: 2 len: 6 to MEMORY</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO reduce.InMemoryMapOutput: Read 2 bytes from map-output for attempt_local1735263799_0001_m_000024_0</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 2, inMemoryMapOutputs.size() -> 26, commitMemory -> 389, usedMemory ->391</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO reduce.LocalFetcher: localfetcher#1 about to shuffle output of map attempt_local1735263799_0001_m_000011_0 decomp: 2 len: 6 to MEMORY</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO reduce.InMemoryMapOutput: Read 2 bytes from map-output for attempt_local1735263799_0001_m_000011_0</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 2, inMemoryMapOutputs.size() -> 27, commitMemory -> 391, usedMemory ->393</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO reduce.LocalFetcher: localfetcher#1 about to shuffle output of map attempt_local1735263799_0001_m_000012_0 decomp: 2 len: 6 to MEMORY</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO reduce.InMemoryMapOutput: Read 2 bytes from map-output for attempt_local1735263799_0001_m_000012_0</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 2, inMemoryMapOutputs.size() -> 28, commitMemory -> 393, usedMemory ->395</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO reduce.LocalFetcher: localfetcher#1 about to shuffle output of map attempt_local1735263799_0001_m_000025_0 decomp: 2 len: 6 to MEMORY</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO reduce.InMemoryMapOutput: Read 2 bytes from map-output for attempt_local1735263799_0001_m_000025_0</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 2, inMemoryMapOutputs.size() -> 29, commitMemory -> 395, usedMemory ->397</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO reduce.EventFetcher: EventFetcher is interrupted.. Returning</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO mapred.LocalJobRunner: 29 / 29 copied.</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO reduce.MergeManagerImpl: finalMerge called with 29 in-memory map-outputs and 0 on-disk map-outputs</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO mapred.Merger: Merging 29 sorted segments</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO mapred.Merger: Down to the last merge-pass, with 6 segments left of total size: 241 bytes</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO reduce.MergeManagerImpl: Merged 29 segments, 397 bytes to disk to satisfy reduce memory limit</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO reduce.MergeManagerImpl: Merging 1 files, 345 bytes from disk</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO reduce.MergeManagerImpl: Merging 0 segments, 0 bytes from memory into reduce</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO mapred.Merger: Merging 1 sorted segments</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO mapred.Merger: Down to the last merge-pass, with 1 segments left of total size: 310 bytes</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO mapred.LocalJobRunner: 29 / 29 copied.</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO Configuration.deprecation: mapred.skip.on is deprecated. Instead, use mapreduce.job.skiprecords</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO mapred.Task: Task:attempt_local1735263799_0001_r_000000_0 is done. And is in the process of committing</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO mapred.LocalJobRunner: 29 / 29 copied.</span>
<span style="color: #888888;">16/08/28 12:27:08 INFO mapred.Task: Task attempt_local1735263799_0001_r_000000_0 is allowed to commit now</span>
<span style="color: #888888;">16/08/28 12:27:09 INFO output.FileOutputCommitter: Saved output of task 'attempt_local1735263799_0001_r_000000_0' to hdfs://localhost:9000/user/hadoop/grep-temp-1779120689/_temporary/0/task_local1735263799_0001_r_000000</span>
<span style="color: #888888;">16/08/28 12:27:09 INFO mapred.LocalJobRunner: reduce > reduce</span>
<span style="color: #888888;">16/08/28 12:27:09 INFO mapred.Task: Task 'attempt_local1735263799_0001_r_000000_0' done.</span>
<span style="color: #888888;">16/08/28 12:27:09 INFO mapred.LocalJobRunner: Finishing task: attempt_local1735263799_0001_r_000000_0</span>
<span style="color: #888888;">16/08/28 12:27:09 INFO mapred.LocalJobRunner: reduce task executor complete.</span>
<span style="color: #888888;">16/08/28 12:27:09 INFO mapreduce.Job: map 100% reduce 100%</span>
<span style="color: #888888;">16/08/28 12:27:10 INFO mapreduce.Job: Job job_local1735263799_0001 completed successfully</span>
<span style="color: #888888;">16/08/28 12:27:10 INFO mapreduce.Job: Counters: 35</span>
<span style="color: #888888;"> File System Counters</span>
<span style="color: #888888;"> FILE: Number of bytes read=10112916</span>
<span style="color: #888888;"> FILE: Number of bytes written=17653982</span>
<span style="color: #888888;"> FILE: Number of read operations=0</span>
<span style="color: #888888;"> FILE: Number of large read operations=0</span>
<span style="color: #888888;"> FILE: Number of write operations=0</span>
<span style="color: #888888;"> HDFS: Number of bytes read=1763191</span>
<span style="color: #888888;"> HDFS: Number of bytes written=437</span>
<span style="color: #888888;"> HDFS: Number of read operations=1021</span>
<span style="color: #888888;"> HDFS: Number of large read operations=0</span>
<span style="color: #888888;"> HDFS: Number of write operations=32</span>
<span style="color: #888888;"> Map-Reduce Framework</span>
<span style="color: #888888;"> Map input records=2069</span>
<span style="color: #888888;"> Map output records=24</span>
<span style="color: #888888;"> Map output bytes=590</span>
<span style="color: #888888;"> Map output materialized bytes=513</span>
<span style="color: #888888;"> Input split bytes=3534</span>
<span style="color: #888888;"> Combine input records=24</span>
<span style="color: #888888;"> Combine output records=13</span>
<span style="color: #888888;"> Reduce input groups=11</span>
<span style="color: #888888;"> Reduce shuffle bytes=513</span>
<span style="color: #888888;"> Reduce input records=13</span>
<span style="color: #888888;"> Reduce output records=11</span>
<span style="color: #888888;"> Spilled Records=26</span>
<span style="color: #888888;"> Shuffled Maps =29</span>
<span style="color: #888888;"> Failed Shuffles=0</span>
<span style="color: #888888;"> Merged Map outputs=29</span>
<span style="color: #888888;"> GC time elapsed (ms)=110</span>
<span style="color: #888888;"> Total committed heap usage (bytes)=15204876288</span>
<span style="color: #888888;"> Shuffle Errors</span>
<span style="color: #888888;"> BAD_ID=0</span>
<span style="color: #888888;"> CONNECTION=0</span>
<span style="color: #888888;"> IO_ERROR=0</span>
<span style="color: #888888;"> WRONG_LENGTH=0</span>
<span style="color: #888888;"> WRONG_MAP=0</span>
<span style="color: #888888;"> WRONG_REDUCE=0</span>
<span style="color: #888888;"> File Input Format Counters </span>
<span style="color: #888888;"> Bytes Read=76859</span>
<span style="color: #888888;"> File Output Format Counters </span>
<span style="color: #888888;"> Bytes Written=437</span>
<span style="color: #888888;">16/08/28 12:27:10 INFO jvm.JvmMetrics: Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized</span>
<span style="color: #888888;">16/08/28 12:27:10 INFO input.FileInputFormat: Total input paths to process : 1</span>
<span style="color: #888888;">16/08/28 12:27:10 INFO mapreduce.JobSubmitter: number of splits:1</span>
<span style="color: #888888;">16/08/28 12:27:10 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_local1915471944_0002</span>
<span style="color: #888888;">16/08/28 12:27:10 INFO mapreduce.Job: The url to track the job: http://localhost:8080/</span>
<span style="color: #888888;">16/08/28 12:27:10 INFO mapreduce.Job: Running job: job_local1915471944_0002</span>
<span style="color: #888888;">16/08/28 12:27:10 INFO mapred.LocalJobRunner: OutputCommitter set in config null</span>
<span style="color: #888888;">16/08/28 12:27:10 INFO output.FileOutputCommitter: File Output Committer Algorithm version is 1</span>
<span style="color: #888888;">16/08/28 12:27:10 INFO mapred.LocalJobRunner: OutputCommitter is org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter</span>
<span style="color: #888888;">16/08/28 12:27:10 INFO mapred.LocalJobRunner: Waiting for map tasks</span>
<span style="color: #888888;">16/08/28 12:27:10 INFO mapred.LocalJobRunner: Starting task: attempt_local1915471944_0002_m_000000_0</span>
<span style="color: #888888;">16/08/28 12:27:10 INFO output.FileOutputCommitter: File Output Committer Algorithm version is 1</span>
<span style="color: #888888;">16/08/28 12:27:10 INFO mapred.Task: Using ResourceCalculatorProcessTree : [ ]</span>
<span style="color: #888888;">16/08/28 12:27:10 INFO mapred.MapTask: Processing split: hdfs://localhost:9000/user/hadoop/grep-temp-1779120689/part-r-00000:0+437</span>
<span style="color: #888888;">16/08/28 12:27:10 INFO mapred.MapTask: (EQUATOR) 0 kvi 26214396(104857584)</span>
<span style="color: #888888;">16/08/28 12:27:10 INFO mapred.MapTask: mapreduce.task.io.sort.mb: 100</span>
<span style="color: #888888;">16/08/28 12:27:10 INFO mapred.MapTask: soft limit at 83886080</span>
<span style="color: #888888;">16/08/28 12:27:10 INFO mapred.MapTask: bufstart = 0; bufvoid = 104857600</span>
<span style="color: #888888;">16/08/28 12:27:10 INFO mapred.MapTask: kvstart = 26214396; length = 6553600</span>
<span style="color: #888888;">16/08/28 12:27:10 INFO mapred.MapTask: Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer</span>
<span style="color: #888888;">16/08/28 12:27:10 INFO mapred.LocalJobRunner: </span>
<span style="color: #888888;">16/08/28 12:27:10 INFO mapred.MapTask: Starting flush of map output</span>
<span style="color: #888888;">16/08/28 12:27:10 INFO mapred.MapTask: Spilling map output</span>
<span style="color: #888888;">16/08/28 12:27:10 INFO mapred.MapTask: bufstart = 0; bufend = 263; bufvoid = 104857600</span>
<span style="color: #888888;">16/08/28 12:27:10 INFO mapred.MapTask: kvstart = 26214396(104857584); kvend = 26214356(104857424); length = 41/6553600</span>
<span style="color: #888888;">16/08/28 12:27:10 INFO mapred.MapTask: Finished spill 0</span>
<span style="color: #888888;">16/08/28 12:27:10 INFO mapred.Task: Task:attempt_local1915471944_0002_m_000000_0 is done. And is in the process of committing</span>
<span style="color: #888888;">16/08/28 12:27:10 INFO mapred.LocalJobRunner: map</span>
<span style="color: #888888;">16/08/28 12:27:10 INFO mapred.Task: Task 'attempt_local1915471944_0002_m_000000_0' done.</span>
<span style="color: #888888;">16/08/28 12:27:10 INFO mapred.LocalJobRunner: Finishing task: attempt_local1915471944_0002_m_000000_0</span>
<span style="color: #888888;">16/08/28 12:27:10 INFO mapred.LocalJobRunner: map task executor complete.</span>
<span style="color: #888888;">16/08/28 12:27:10 INFO mapred.LocalJobRunner: Waiting for reduce tasks</span>
<span style="color: #888888;">16/08/28 12:27:10 INFO mapred.LocalJobRunner: Starting task: attempt_local1915471944_0002_r_000000_0</span>
<span style="color: #888888;">16/08/28 12:27:10 INFO output.FileOutputCommitter: File Output Committer Algorithm version is 1</span>
<span style="color: #888888;">16/08/28 12:27:10 INFO mapred.Task: Using ResourceCalculatorProcessTree : [ ]</span>
<span style="color: #888888;">16/08/28 12:27:10 INFO mapred.ReduceTask: Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@7efca19d</span>
<span style="color: #888888;">16/08/28 12:27:10 INFO reduce.MergeManagerImpl: MergerManager: memoryLimit=371405600, maxSingleShuffleLimit=92851400, mergeThreshold=245127712, ioSortFactor=10, memToMemMergeOutputsThreshold=10</span>
<span style="color: #888888;">16/08/28 12:27:10 INFO reduce.EventFetcher: attempt_local1915471944_0002_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events</span>
<span style="color: #888888;">16/08/28 12:27:10 INFO reduce.LocalFetcher: localfetcher#2 about to shuffle output of map attempt_local1915471944_0002_m_000000_0 decomp: 287 len: 291 to MEMORY</span>
<span style="color: #888888;">16/08/28 12:27:10 INFO reduce.InMemoryMapOutput: Read 287 bytes from map-output for attempt_local1915471944_0002_m_000000_0</span>
<span style="color: #888888;">16/08/28 12:27:10 INFO reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 287, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->287</span>
<span style="color: #888888;">16/08/28 12:27:10 INFO reduce.EventFetcher: EventFetcher is interrupted.. Returning</span>
<span style="color: #888888;">16/08/28 12:27:10 INFO mapred.LocalJobRunner: 1 / 1 copied.</span>
<span style="color: #888888;">16/08/28 12:27:10 INFO reduce.MergeManagerImpl: finalMerge called with 1 in-memory map-outputs and 0 on-disk map-outputs</span>
<span style="color: #888888;">16/08/28 12:27:10 INFO mapred.Merger: Merging 1 sorted segments</span>
<span style="color: #888888;">16/08/28 12:27:10 INFO mapred.Merger: Down to the last merge-pass, with 1 segments left of total size: 277 bytes</span>
<span style="color: #888888;">16/08/28 12:27:10 INFO reduce.MergeManagerImpl: Merged 1 segments, 287 bytes to disk to satisfy reduce memory limit</span>
<span style="color: #888888;">16/08/28 12:27:10 INFO reduce.MergeManagerImpl: Merging 1 files, 291 bytes from disk</span>
<span style="color: #888888;">16/08/28 12:27:10 INFO reduce.MergeManagerImpl: Merging 0 segments, 0 bytes from memory into reduce</span>
<span style="color: #888888;">16/08/28 12:27:10 INFO mapred.Merger: Merging 1 sorted segments</span>
<span style="color: #888888;">16/08/28 12:27:10 INFO mapred.Merger: Down to the last merge-pass, with 1 segments left of total size: 277 bytes</span>
<span style="color: #888888;">16/08/28 12:27:10 INFO mapred.LocalJobRunner: 1 / 1 copied.</span>
<span style="color: #888888;">16/08/28 12:27:10 INFO mapred.Task: Task:attempt_local1915471944_0002_r_000000_0 is done. And is in the process of committing</span>
<span style="color: #888888;">16/08/28 12:27:10 INFO mapred.LocalJobRunner: 1 / 1 copied.</span>
<span style="color: #888888;">16/08/28 12:27:10 INFO mapred.Task: Task attempt_local1915471944_0002_r_000000_0 is allowed to commit now</span>
<span style="color: #888888;">16/08/28 12:27:10 INFO output.FileOutputCommitter: Saved output of task 'attempt_local1915471944_0002_r_000000_0' to hdfs://localhost:9000/user/hadoop/output/_temporary/0/task_local1915471944_0002_r_000000</span>
<span style="color: #888888;">16/08/28 12:27:10 INFO mapred.LocalJobRunner: reduce > reduce</span>
<span style="color: #888888;">16/08/28 12:27:10 INFO mapred.Task: Task 'attempt_local1915471944_0002_r_000000_0' done.</span>
<span style="color: #888888;">16/08/28 12:27:10 INFO mapred.LocalJobRunner: Finishing task: attempt_local1915471944_0002_r_000000_0</span>
<span style="color: #888888;">16/08/28 12:27:10 INFO mapred.LocalJobRunner: reduce task executor complete.</span>
<span style="color: #888888;">16/08/28 12:27:11 INFO mapreduce.Job: Job job_local1915471944_0002 running in uber mode : false</span>
<span style="color: #888888;">16/08/28 12:27:11 INFO mapreduce.Job: map 100% reduce 100%</span>
<span style="color: #888888;">16/08/28 12:27:11 INFO mapreduce.Job: Job job_local1915471944_0002 completed successfully</span>
<span style="color: #888888;">16/08/28 12:27:11 INFO mapreduce.Job: Counters: 35</span>
<span style="color: #888888;"> File System Counters</span>
<span style="color: #888888;"> FILE: Number of bytes read=1311058</span>
<span style="color: #888888;"> FILE: Number of bytes written=2343873</span>
<span style="color: #888888;"> FILE: Number of read operations=0</span>
<span style="color: #888888;"> FILE: Number of large read operations=0</span>
<span style="color: #888888;"> FILE: Number of write operations=0</span>
<span style="color: #888888;"> HDFS: Number of bytes read=154592</span>
<span style="color: #888888;"> HDFS: Number of bytes written=1071</span>
<span style="color: #888888;"> HDFS: Number of read operations=151</span>
<span style="color: #888888;"> HDFS: Number of large read operations=0</span>
<span style="color: #888888;"> HDFS: Number of write operations=16</span>
<span style="color: #888888;"> Map-Reduce Framework</span>
<span style="color: #888888;"> Map input records=11</span>
<span style="color: #888888;"> Map output records=11</span>
<span style="color: #888888;"> Map output bytes=263</span>
<span style="color: #888888;"> Map output materialized bytes=291</span>
<span style="color: #888888;"> Input split bytes=132</span>
<span style="color: #888888;"> Combine input records=0</span>
<span style="color: #888888;"> Combine output records=0</span>
<span style="color: #888888;"> Reduce input groups=5</span>
<span style="color: #888888;"> Reduce shuffle bytes=291</span>
<span style="color: #888888;"> Reduce input records=11</span>
<span style="color: #888888;"> Reduce output records=11</span>
<span style="color: #888888;"> Spilled Records=22</span>
<span style="color: #888888;"> Shuffled Maps =1</span>
<span style="color: #888888;"> Failed Shuffles=0</span>
<span style="color: #888888;"> Merged Map outputs=1</span>
<span style="color: #888888;"> GC time elapsed (ms)=0</span>
<span style="color: #888888;"> Total committed heap usage (bytes)=1061158912</span>
<span style="color: #888888;"> Shuffle Errors</span>
<span style="color: #888888;"> BAD_ID=0</span>
<span style="color: #888888;"> CONNECTION=0</span>
<span style="color: #888888;"> IO_ERROR=0</span>
<span style="color: #888888;"> WRONG_LENGTH=0</span>
<span style="color: #888888;"> WRONG_MAP=0</span>
<span style="color: #888888;"> WRONG_REDUCE=0</span>
<span style="color: #888888;"> File Input Format Counters </span>
<span style="color: #888888;"> Bytes Read=437</span>
<span style="color: #888888;"> File Output Format Counters </span>
<span style="color: #888888;"> Bytes Written=197</span>
<span style="color: #c65d09; font-weight: bold;">hadoop@637c83896b9d:~/hadoop-2.7.3$</span>
<span style="color: #c65d09; font-weight: bold;">hadoop@637c83896b9d:~/hadoop-2.7.3$</span>
<span style="color: #c65d09; font-weight: bold;">hadoop@637c83896b9d:~/hadoop-2.7.3$</span>
<span style="color: #c65d09; font-weight: bold;">hadoop@637c83896b9d:~/hadoop-2.7.3$</span> bin/hdfs dfs -get output output
<span style="color: #c65d09; font-weight: bold;">hadoop@637c83896b9d:~/hadoop-2.7.3$</span> cat output/*
<span style="color: #888888;">cat: output/output: Is a directory</span>
<span style="color: #888888;">1 dfsadmin</span>
<span style="color: #c65d09; font-weight: bold;">hadoop@637c83896b9d:~/hadoop-2.7.3$</span>
</pre>
</div>
<br />
<!-- HTML generated using hilite.me --><br />
<div style="background: #ffffff; border-width: 0.1em 0.1em 0.1em 0.8em; border: solid gray; overflow: auto; padding: 0.2em 0.6em; width: auto;">
<pre style="line-height: 125%; margin: 0;"><span style="color: #c65d09; font-weight: bold;">hadoop@637c83896b9d:~/hadoop-2.7.3$</span> sbin/stop-dfs.sh
<span style="color: #888888;">Stopping namenodes on [localhost]</span>
<span style="color: #888888;">localhost: stopping namenode</span>
<span style="color: #888888;">localhost: stopping datanode</span>
<span style="color: #888888;">Stopping secondary namenodes [0.0.0.0]</span>
<span style="color: #888888;">0.0.0.0: stopping secondarynamenode</span>
<span style="color: #c65d09; font-weight: bold;">hadoop@637c83896b9d:~/hadoop-2.7.3$</span>
</pre>
</div>
<br />
<br />
Commit container and push to hub<br />
<br />
<!-- HTML generated using hilite.me --><br />
<div style="background: #ffffff; border-width: 0.1em 0.1em 0.1em 0.8em; border: solid gray; overflow: auto; padding: 0.2em 0.6em; width: auto;">
<pre style="line-height: 125%; margin: 0;"><span style="color: #c65d09; font-weight: bold;">ubuntu@node2:~$</span> docker commit 637c83896b9d nutthaphon/hbase:hdfs
<span style="color: #888888;">sha256:e3dc0f939d25f633e4bc6a5f1fe395b64736e81a5cf7cc3208d4ce7006983fe0</span>
<span style="color: #c65d09; font-weight: bold;">ubuntu@node2:~$</span> docker login
<span style="color: #888888;">Login with your Docker ID to push and pull images from Docker Hub. If you don't have a Docker ID, head over to https://hub.docker.com to create one.</span>
<span style="color: #888888;">Username (nutthaphon): </span>
<span style="color: #888888;">Password: </span>
<span style="color: #888888;">Login Succeeded</span>
<span style="color: #c65d09; font-weight: bold;">ubuntu@node2:~$</span> docker push nutthaphon/hbase:hdfs
<span style="color: #888888;">The push refers to a repository [docker.io/nutthaphon/hbase]</span>
<span style="color: #888888;">25c04ffed27a: Pushed </span>
<span style="color: #888888;">01e76fc6590f: Layer already exists </span>
<span style="color: #888888;">447f88c8358f: Layer already exists </span>
<span style="color: #888888;">df9a135a6949: Layer already exists </span>
<span style="color: #888888;">dbaa8ea1faf9: Layer already exists </span>
<span style="color: #888888;">8a14f84e5837: Layer already exists </span>
<span style="color: #888888;">hdfs: digest: sha256:2436da12eadc964f8389ec7b1c868563ec53b7af901d8fb179afb7d72df814ac size: 1578</span>
<span style="color: #c65d09; font-weight: bold;">ubuntu@node2:~$</span>
</pre>
</div>
<br />
<br />
Next <a href="http://nutthaphon.blogspot.com/2016/08/apache-hbase-pseudo-distributed-part-ii.html" target="_blank">Apache HBase - Pseudo-distributed (Part II)</a><br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />พี่นัทhttp://www.blogger.com/profile/14291406590594822628noreply@blogger.com0tag:blogger.com,1999:blog-2488490746136830746.post-59718622242090828952016-08-26T22:48:00.002+07:002016-08-27T21:27:39.588+07:00Apache HBase - Standalone mode (Part I)<br />
This is the default mode. In standalone mode, HBase does not use HDFS -- it uses the local filesystem instead -- and it runs all HBase daemons and a local <span style="color: blue;"><a href="https://zookeeper.apache.org/" target="_blank">ZooKeeper</a></span> all up in the same JVM. Zookeeper binds to a well known port so clients may talk to HBase.<br />
<br />
<h3>
Prepare Docker container</h3>
<div style="background: #ffffff; border-width: 0.1em 0.1em 0.1em 0.8em; border: solid gray; overflow: auto; padding: 0.2em 0.6em; width: auto;">
<pre style="line-height: 125%; margin: 0;"><span style="color: #c65d09; font-weight: bold;">ubuntu@node2:~$</span> docker network create <span style="background-color: #fff0f0; color: #666666; font-weight: bold;">\</span>
<span style="color: #c65d09; font-weight: bold;">></span> --driver<span style="color: #333333;">=</span>bridge <span style="background-color: #fff0f0; color: #666666; font-weight: bold;">\</span>
<span style="color: #c65d09; font-weight: bold;">></span> --subnet<span style="color: #333333;">=</span>172.29.0.0/16 <span style="background-color: #fff0f0; color: #666666; font-weight: bold;">\</span>
<span style="color: #c65d09; font-weight: bold;">></span> --ip-range<span style="color: #333333;">=</span>172.29.5.0/24 <span style="background-color: #fff0f0; color: #666666; font-weight: bold;">\</span>
<span style="color: #c65d09; font-weight: bold;">></span> --gateway<span style="color: #333333;">=</span>172.29.5.254 <span style="background-color: #fff0f0; color: #666666; font-weight: bold;">\</span>
<span style="color: #c65d09; font-weight: bold;">></span> my-net
<span style="color: #888888;">6f5295c3ac2c7aa3c980e5ebd9140714f31000ef483d07aa1c8c6162bea12e5a</span>
<span style="color: #c65d09; font-weight: bold;">ubuntu@node2:~$</span>
<span style="color: #c65d09; font-weight: bold;">ubuntu@node2:~$</span> docker network ls
<span style="color: #888888;">NETWORK ID NAME DRIVER SCOPE</span>
<span style="color: #888888;">38916a17da9f bridge bridge local </span>
<span style="color: #888888;">844343d53939 host host local </span>
<span style="color: #888888;">6f5295c3ac2c my-net bridge local </span>
<span style="color: #888888;">8150faa4b0d4 none null local </span>
<span style="color: #c65d09; font-weight: bold;">ubuntu@node2:~$</span> docker network inspect my-net
<span style="color: #888888;">[</span>
<span style="color: #888888;"> {</span>
<span style="color: #888888;"> "Name": "my-net",</span>
<span style="color: #888888;"> "Id": "6f5295c3ac2c7aa3c980e5ebd9140714f31000ef483d07aa1c8c6162bea12e5a",</span>
<span style="color: #888888;"> "Scope": "local",</span>
<span style="color: #888888;"> "Driver": "bridge",</span>
<span style="color: #888888;"> "EnableIPv6": false,</span>
<span style="color: #888888;"> "IPAM": {</span>
<span style="color: #888888;"> "Driver": "default",</span>
<span style="color: #888888;"> "Options": {},</span>
<span style="color: #888888;"> "Config": [</span>
<span style="color: #888888;"> {</span>
<span style="color: #888888;"> "Subnet": "172.29.0.0/16",</span>
<span style="color: #888888;"> "IPRange": "172.29.5.0/24",</span>
<span style="color: #888888;"> "Gateway": "172.29.5.254"</span>
<span style="color: #888888;"> }</span>
<span style="color: #888888;"> ]</span>
<span style="color: #888888;"> },</span>
<span style="color: #888888;"> "Internal": false,</span>
<span style="color: #888888;"> "Containers": {},</span>
<span style="color: #888888;"> "Options": {},</span>
<span style="color: #888888;"> "Labels": {}</span>
<span style="color: #888888;"> }</span>
<span style="color: #888888;">]</span>
<span style="color: #c65d09; font-weight: bold;">ubuntu@node2:~$</span> docker run --name hbase -it -P <span style="background-color: #fff0f0; color: #666666; font-weight: bold;">\</span>
<span style="color: #c65d09; font-weight: bold;">></span> --net<span style="color: #333333;">=</span><span style="background-color: #fff0f0;">"my-net"</span> <span style="background-color: #fff0f0; color: #666666; font-weight: bold;">\</span>
<span style="color: #c65d09; font-weight: bold;">></span> --mac-address<span style="color: #333333;">=</span><span style="background-color: #fff0f0;">"52:1d:44:ff:b8:40"</span> <span style="background-color: #fff0f0; color: #666666; font-weight: bold;">\</span>
<span style="color: #c65d09; font-weight: bold;">></span> --add-host<span style="color: #333333;">=</span><span style="background-color: #fff0f0;">"hbase:172.29.5.1"</span> <span style="background-color: #fff0f0; color: #666666; font-weight: bold;">\</span>
<span style="color: #c65d09; font-weight: bold;">></span> --ip<span style="color: #333333;">=</span><span style="background-color: #fff0f0;">"172.29.5.1"</span> ubuntu:14.04 bash
<span style="color: #888888;">148087c38a7fa0297629a5701d958978b3f537d1f5d9b74ae049e0cc91f1836d</span>
<span style="color: #c65d09; font-weight: bold;">root@0fb87ce32de3:/#</span>
</pre>
</div>
<br />
<br />
<br />
<br />
<h3>
Install JDK 8 (instruction from <a href="http://www.webupd8.org/2012/09/install-oracle-java-8-in-ubuntu-via-ppa.html" target="_blank">here</a>)</h3>
$ sudo apt-get install python-software-properties<br />
$ sudo apt-get install software-properties-common<br />
$ sudo add-apt-repository ppa:webupd8team/java<br />
$ sudo apt-get update<br />
<br />
$ sudo apt-get install oracle-java8-installer<br />
<br />
-- change java when has more than one Java Home<br />
$ sudo update-alternatives --config java<br />
<br />
<i><span style="color: blue;"><br /></span></i>
<i><span style="color: blue;">--- COMMIT first stage after install JDK</span></i><br />
<i><span style="color: blue;">--- $ docker commit 0fb87ce32de3 hbase:install_jdk8</span></i><br />
<br />
Now we use software folder from host directory as a mount point in container. So we run new container with a latest committed image ( <span style="color: blue;">hbase:install_jdk8 </span>).<br />
<br />
<!-- HTML generated using hilite.me --><br />
<h3>
Install HBase</h3>
<div style="background: #ffffff; border-width: 0.1em 0.1em 0.1em 0.8em; border: solid gray; overflow: auto; padding: 0.2em 0.6em; width: auto;">
<pre style="line-height: 125%; margin: 0;"><span style="color: #c65d09; font-weight: bold;">ubuntu@node2:~/software$</span> docker run --name hbase -it -P <span style="background-color: #fff0f0; color: #666666; font-weight: bold;">\</span>
<span style="color: #c65d09; font-weight: bold;">></span> --net<span style="color: #333333;">=</span><span style="background-color: #fff0f0;">"my-net"</span> <span style="background-color: #fff0f0; color: #666666; font-weight: bold;">\</span>
<span style="color: #c65d09; font-weight: bold;">></span> --mac-address<span style="color: #333333;">=</span><span style="background-color: #fff0f0;">"52:1d:44:ff:b8:40"</span> <span style="background-color: #fff0f0; color: #666666; font-weight: bold;">\</span>
<span style="color: #c65d09; font-weight: bold;">></span> --add-host<span style="color: #333333;">=</span><span style="background-color: #fff0f0;">"hbase:172.29.5.1"</span> <span style="background-color: #fff0f0; color: #666666; font-weight: bold;">\</span>
<span style="color: #c65d09; font-weight: bold;">></span> --ip<span style="color: #333333;">=</span><span style="background-color: #fff0f0;">"172.29.5.1"</span> <span style="background-color: #fff0f0; color: #666666; font-weight: bold;">\</span>
<span style="color: #c65d09; font-weight: bold;">></span> -v /home/ubuntu/software:/software <span style="background-color: #fff0f0; color: #666666; font-weight: bold;">\</span>
<span style="color: #c65d09; font-weight: bold;">></span> hbase:install_jdk8 bash
<span style="color: #c65d09; font-weight: bold;">root@637c83896b9d:/#</span>
<span style="color: #c65d09; font-weight: bold;">root@637c83896b9d:/#</span>
<span style="color: #c65d09; font-weight: bold;">root@637c83896b9d:/#</span> df -k
<span style="color: #888888;">Filesystem 1K-blocks Used Available Use% Mounted on</span>
<span style="color: #888888;">none 103080204 13881500 83939496 15% /</span>
<span style="color: #888888;">tmpfs 5120624 0 5120624 0% /dev</span>
<span style="color: #888888;">tmpfs 5120624 0 5120624 0% /sys/fs/cgroup</span>
<span style="color: #888888;">/dev/sda1 103080204 13881500 83939496 15% /software</span>
<span style="color: #888888;">shm 65536 0 65536 0% /dev/shm</span>
<span style="color: #c65d09; font-weight: bold;">root@637c83896b9d:/#</span>
<span style="color: #c65d09; font-weight: bold;">root@637c83896b9d:/#</span>
<span style="color: #c65d09; font-weight: bold;">root@637c83896b9d:/#</span>
<span style="color: #c65d09; font-weight: bold;">root@637c83896b9d:/#</span> java -version
<span style="color: #888888;">java version "1.8.0_101"</span>
<span style="color: #888888;">Java(TM) SE Runtime Environment (build 1.8.0_101-b13)</span>
<span style="color: #888888;">Java HotSpot(TM) 64-Bit Server VM (build 25.101-b13, mixed mode)</span>
<span style="color: #c65d09; font-weight: bold;">root@637c83896b9d:/#</span>
<span style="color: #c65d09; font-weight: bold;">root@637c83896b9d:/#</span> useradd hbase -m -s /bin/bash
</pre>
<pre style="line-height: 125%; margin: 0;"><pre style="line-height: 125%; margin: 0;"><span style="color: #c65d09; font-weight: bold;">root@637c83896b9d:/# </span><span style="line-height: 125%;">exit</span></pre>
</pre>
</div>
<br />
Logon with non-root user<br />
<br />
<!-- HTML generated using hilite.me --><br />
<div style="background: #ffffff; border-width: 0.1em 0.1em 0.1em 0.8em; border: solid gray; overflow: auto; padding: 0.2em 0.6em; width: auto;">
<pre style="line-height: 125%; margin: 0;"><span style="color: #c65d09; font-weight: bold;">ubuntu@node2:~/software$</span> docker start hbase
<span style="color: #888888;">hbase</span>
<span style="color: #c65d09; font-weight: bold;">ubuntu@node2:~/software$</span> docker <span style="color: #007020;">exec</span> -it --user hbase hbase bash
<span style="color: #c65d09; font-weight: bold;">hbase@637c83896b9d:/$</span>
</pre>
</div>
<br />
<!-- HTML generated using hilite.me --><br />
<div style="background: #ffffff; border-width: 0.1em 0.1em 0.1em 0.8em; border: solid gray; overflow: auto; padding: 0.2em 0.6em; width: auto;">
<pre style="line-height: 125%; margin: 0;"><span style="color: #c65d09; font-weight: bold;">hbase@637c83896b9d:~$</span> tar xzvf /software/hbase-1.2.2-bin.tar.gz
<span style="color: #c65d09; font-weight: bold;">hbase@637c83896b9d:~$</span> <span style="color: #007020;">cd </span>hbase-1.2.2/
<span style="color: #c65d09; font-weight: bold;">hbase@637c83896b9d:~/hbase-1.2.2$</span> ls
<span style="color: #888888;">CHANGES.txt LEGAL LICENSE.txt NOTICE.txt README.txt bin conf docs hbase-webapps lib</span>
</pre>
</div>
<br />
<!-- HTML generated using hilite.me --><br />
<div style="background: #ffffff; border-width: 0.1em 0.1em 0.1em 0.8em; border: solid gray; overflow: auto; padding: 0.2em 0.6em; width: auto;">
<pre style="line-height: 125%; margin: 0;"><span style="color: #c65d09; font-weight: bold;">hbase@637c83896b9d:~/hbase-1.2.2/conf$</span> cat hbase-site.xml
<span style="color: #888888;"><?xml version="1.0"?></span>
<span style="color: #888888;"><?xml-stylesheet type="text/xsl" href="configuration.xsl"?></span>
<span style="color: #888888;"><!--</span>
<span style="color: #888888;">/**</span>
<span style="color: #888888;"> *</span>
<span style="color: #888888;"> * Licensed to the Apache Software Foundation (ASF) under one</span>
<span style="color: #888888;"> * or more contributor license agreements. See the NOTICE file</span>
<span style="color: #888888;"> * distributed with this work for additional information</span>
<span style="color: #888888;"> * regarding copyright ownership. The ASF licenses this file</span>
<span style="color: #888888;"> * to you under the Apache License, Version 2.0 (the</span>
<span style="color: #888888;"> * "License"); you may not use this file except in compliance</span>
<span style="color: #888888;"> * with the License. You may obtain a copy of the License at</span>
<span style="color: #888888;"> *</span>
<span style="color: #888888;"> * http://www.apache.org/licenses/LICENSE-2.0</span>
<span style="color: #888888;"> *</span>
<span style="color: #888888;"> * Unless required by applicable law or agreed to in writing, software</span>
<span style="color: #888888;"> * distributed under the License is distributed on an "AS IS" BASIS,</span>
<span style="color: #888888;"> * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.</span>
<span style="color: #888888;"> * See the License for the specific language governing permissions and</span>
<span style="color: #888888;"> * limitations under the License.</span>
<span style="color: #888888;"> */</span>
<span style="color: #888888;">--></span>
<span style="color: #888888;"><configuration></span>
<span style="color: #888888;"><property></span>
<span style="color: #888888;"> <name>hbase.rootdir</name></span>
<span style="color: #888888;"> <value>file:///home/hbase/hbase-1.2.2</value></span>
<span style="color: #888888;"> </property></span>
<span style="color: #888888;"> <property></span>
<span style="color: #888888;"> <name>hbase.zookeeper.property.dataDir</name></span>
<span style="color: #888888;"> <value>/home/hbase/zookeeper</value></span>
<span style="color: #888888;"> </property></span>
<span style="color: #888888;"></configuration></span>
<span style="color: #c65d09; font-weight: bold;">hbase@637c83896b9d:~/hbase-1.2.2/conf$</span> <span style="color: #007020;">cd</span>
</pre>
</div>
<br />
<br />
<!-- HTML generated using hilite.me --><br />
<div style="background: #ffffff; border-width: 0.1em 0.1em 0.1em 0.8em; border: solid gray; overflow: auto; padding: 0.2em 0.6em; width: auto;">
<pre style="line-height: 125%; margin: 0;"><span style="color: #c65d09; font-weight: bold;">hbase@637c83896b9d:~/hbase-1.2.2/logs$</span> <span style="color: #007020;">echo</span> <span style="color: #996633;">$JAVA_HOME</span>
<span style="color: #c65d09; font-weight: bold;">hbase@637c83896b9d:~/hbase-1.2.2/logs$</span> update-alternatives --config java
<span style="color: #888888;">There is only one alternative in link group java (providing /usr/bin/java): /usr/lib/jvm/java-8-oracle/jre/bin/java</span>
<span style="color: #888888;">Nothing to configure.</span>
<span style="color: #c65d09; font-weight: bold;">hbase@637c83896b9d:~/hbase-1.2.2/logs$</span>
<span style="color: #c65d09; font-weight: bold;">hbase@637c83896b9d:~/hbase-1.2.2/logs$</span> <span style="color: #996633;">JAVA_HOME</span><span style="color: #333333;">=</span>/usr/lib/jvm/java-8-oracle
<span style="color: #c65d09; font-weight: bold;">hbase@637c83896b9d:~/hbase-1.2.2/logs$</span> <span style="color: #007020;">export </span>JAVA_HOME
<span style="color: #c65d09; font-weight: bold;">hbase@637c83896b9d:~/hbase-1.2.2/logs$</span>
<span style="color: #c65d09; font-weight: bold;">hbase@637c83896b9d:~/hbase-1.2.2/logs$</span>
<span style="color: #c65d09; font-weight: bold;">hbase@637c83896b9d:~/hbase-1.2.2/logs$</span> <span style="color: #007020;">cd</span> ../bin
<span style="color: #c65d09; font-weight: bold;">hbase@637c83896b9d:~/hbase-1.2.2/bin$</span> ls
<span style="color: #888888;">draining_servers.rb hbase-common.sh hbase-jruby master-backup.sh rolling-restart.sh stop-hbase.sh</span>
<span style="color: #888888;">get-active-master.rb hbase-config.cmd hbase.cmd region_mover.rb shutdown_regionserver.rb test</span>
<span style="color: #888888;">graceful_stop.sh hbase-config.sh hirb.rb region_status.rb start-hbase.cmd thread-pool.rb</span>
<span style="color: #888888;">hbase hbase-daemon.sh local-master-backup.sh regionservers.sh start-hbase.sh zookeepers.sh</span>
<span style="color: #888888;">hbase-cleanup.sh hbase-daemons.sh local-regionservers.sh replication stop-hbase.cmd</span>
<span style="color: #c65d09; font-weight: bold;">hbase@637c83896b9d:~/hbase-1.2.2/bin$</span> ./stop-hbase.sh
<span style="color: #888888;">stopping hbase..................</span>
<span style="color: #c65d09; font-weight: bold;">hbase@637c83896b9d:~/hbase-1.2.2/bin$</span> ./start-hbase.sh
<span style="color: #888888;">starting master, logging to /home/hbase/hbase-1.2.2/bin/../logs/hbase--master-637c83896b9d.out</span>
<span style="color: #888888;">Java HotSpot(TM) 64-Bit Server VM warning: ignoring option PermSize=128m; support was removed in 8.0</span>
<span style="color: #888888;">Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=128m; support was removed in 8.0</span>
<span style="color: #c65d09; font-weight: bold;">hbase@637c83896b9d:~/hbase-1.2.2/bin$</span>
</pre>
</div>
<br />
<br />
-- COMMIT container as stage two ( <span style="color: blue;">hbase:standalone</span> )<br />
<br />
<!-- HTML generated using hilite.me --><br />
<div style="background: #ffffff; border-width: 0.1em 0.1em 0.1em 0.8em; border: solid gray; overflow: auto; padding: 0.2em 0.6em; width: auto;">
<pre style="line-height: 125%; margin: 0;"><span style="color: #c65d09; font-weight: bold;">hbase@637c83896b9d:~/hbase-1.2.2/bin$</span> ./stop-hbase.sh
<span style="color: #888888;">stopping hbase...................</span>
<span style="color: #c65d09; font-weight: bold;">hbase@637c83896b9d:~/hbase-1.2.2/bin$</span> <span style="color: #007020;">exit</span>
<span style="color: #888888;">exit</span>
<span style="color: #c65d09; font-weight: bold;">ubuntu@node2:~$</span> docker ps -a
<span style="color: #888888;">CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES</span>
<span style="color: #888888;">637c83896b9d hbase:install_jdk8 "bash" About an hour ago Up 56 minutes hbase</span>
<span style="color: #c65d09; font-weight: bold;">ubuntu@node2:~$</span> docker ps -a
<span style="color: #888888;">CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES</span>
<span style="color: #888888;">637c83896b9d hbase:install_jdk8 "bash" About an hour ago Up 56 minutes hbase</span>
<span style="color: #c65d09; font-weight: bold;">ubuntu@node2:~$</span> docker commit 637c83896b9d hbase:standalone
<span style="color: #888888;">sha256:00c7c409fdc7d291939d3924b8dfa07eb014ec70c0476abdd13c4ebf2690c07e</span>
<span style="color: #c65d09; font-weight: bold;">ubuntu@node2:~$</span> docker ps -a
<span style="color: #888888;">CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES</span>
<span style="color: #888888;">637c83896b9d hbase:install_jdk8 "bash" About an hour ago Up 57 minutes hbase</span>
<span style="color: #c65d09; font-weight: bold;">ubuntu@node2:~$</span> docker images
<span style="color: #888888;">REPOSITORY TAG IMAGE ID CREATED SIZE</span>
<span style="color: #888888;">hbase standalone 00c7c409fdc7 30 seconds ago 1.189 GB</span>
<span style="color: #888888;">hbase install_jdk8 0ab667093b9b 6 hours ago 840.2 MB</span>
<span style="color: #888888;">ubuntu 14.04 ff6011336327 2 weeks ago 188 MB</span>
<span style="color: #888888;">cloudera/quickstart latest 4239cd2958c6 4 months ago 6.336 GB</span>
<span style="color: #888888;">shipyard/docker-proxy latest cfee14e5d6f2 8 months ago 9.468 MB</span>
</pre>
</div>
<br />
Push to Docker Hub<br />
<br />
<!-- HTML generated using hilite.me --><br />
<div style="background: #ffffff; border-width: 0.1em 0.1em 0.1em 0.8em; border: solid gray; overflow: auto; padding: 0.2em 0.6em; width: auto;">
<pre style="line-height: 125%; margin: 0;"><span style="color: #c65d09; font-weight: bold;">ubuntu@node2:~/.docker$</span> docker ps -a
<span style="color: #888888;">CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES</span>
<span style="color: #888888;">637c83896b9d hbase:install_jdk8 "bash" 21 hours ago Up About an hour hbase</span>
<span style="color: #c65d09; font-weight: bold;">ubuntu@node2:~/.docker$</span> docker commit 637c83896b9d nutthaphon/hbase:1.2.2
<span style="color: #888888;">sha256:a01016dd24ec89508d5e93e13b2bc3d3d2a898bf6945fb4b0af8256aec08f82b</span>
<span style="color: #c65d09; font-weight: bold;">ubuntu@node2:~/.docker$</span> docker login
<span style="color: #888888;">Login with your Docker ID to push and pull images from Docker Hub. If you don't have a Docker ID, head over to https://hub.docker.com to create one.</span>
<span style="color: #888888;">Username (nutthaphon): nutthaphon</span>
<span style="color: #888888;">Password: </span>
<span style="color: #888888;">Login Succeeded</span>
<span style="color: #c65d09; font-weight: bold;">ubuntu@node2:~/.docker$</span> docker images
<span style="color: #888888;">REPOSITORY TAG IMAGE ID CREATED SIZE</span>
<span style="color: #888888;">nutthaphon/hbase 1.2.2 a01016dd24ec 45 seconds ago 1.189 GB</span>
<span style="color: #888888;">hbase 1.2.2 00c7c409fdc7 20 hours ago 1.189 GB</span>
<span style="color: #888888;">hbase standalone 00c7c409fdc7 20 hours ago 1.189 GB</span>
<span style="color: #888888;">hbase install_jdk8 0ab667093b9b 26 hours ago 840.2 MB</span>
<span style="color: #888888;">ubuntu 14.04 ff6011336327 2 weeks ago 188 MB</span>
<span style="color: #888888;">cloudera/quickstart latest 4239cd2958c6 4 months ago 6.336 GB</span>
<span style="color: #888888;">shipyard/docker-proxy latest cfee14e5d6f2 8 months ago 9.468 MB</span>
<span style="color: #c65d09; font-weight: bold;">ubuntu@node2:~/.docker$</span> docker push nutthaphon/hbase:1.2.2
<span style="color: #888888;">The push refers to a repository [docker.io/nutthaphon/hbase]</span>
<span style="color: #888888;">cafd3a1794b0: Pushed </span>
<span style="color: #888888;">01e76fc6590f: Pushed </span>
<span style="color: #888888;">447f88c8358f: Pushed </span>
<span style="color: #888888;">df9a135a6949: Pushed </span>
<span style="color: #888888;">dbaa8ea1faf9: Pushed </span>
<span style="color: #888888;">8a14f84e5837: Pushed </span>
<span style="color: #888888;">1.2.2: digest: sha256:917e7387ac064e9c2daa0cdcc63279a39d78e06ef2bb6568558adfd06d172cf1 size: 1578</span>
<span style="color: #c65d09; font-weight: bold;">ubuntu@node2:~/.docker$</span>
</pre>
</div>
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />พี่นัทhttp://www.blogger.com/profile/14291406590594822628noreply@blogger.com0tag:blogger.com,1999:blog-2488490746136830746.post-12367289477729061782016-08-22T23:19:00.000+07:002016-08-26T18:10:13.624+07:00Scrapy Tutorial<br />
<br />
Pre-requirsite<br />
<br />
<!-- HTML generated using hilite.me --><br />
<div style="background: #ffffff; border-width: 0.1em 0.1em 0.1em 0.8em; border: solid gray; overflow: auto; padding: 0.2em 0.6em; width: auto;">
<pre style="line-height: 125%; margin: 0;"><span style="color: #c65d09; font-weight: bold;">#</span>Install Python Package Index
<span style="color: #888888;">sudo apt-get install python-pip</span>
<span style="color: #888888;">pip search scrapy</span>
<span style="color: #888888;">pip install scrapy</span>
<span style="color: #888888;">pip show scrapy</span>
<span style="color: #c65d09; font-weight: bold;">#</span>pip uninstall scrapy
</pre>
</div>
<br />
<br />
An instruction from <a href="http://doc.scrapy.org/en/latest/intro/tutorial.html" target="_blank">Scrapy.org</a><br />
<br />
<!-- HTML generated using hilite.me --><br />
<div style="background: #ffffff; border-width: 0.1em 0.1em 0.1em 0.8em; border: solid gray; overflow: auto; padding: 0.2em 0.6em; width: auto;">
<pre style="line-height: 125%; margin: 0;"><span style="color: #c65d09; font-weight: bold;">nutt@nutt-pc:~/OneDrive/Scrapy$</span> scrapy startproject tutorial
<span style="color: #888888;">New Scrapy project 'tutorial', using template directory '/usr/local/lib/python2.7/dist-packages/scrapy/templates/project', created in:</span>
<span style="color: #888888;"> /home/nutt/OneDrive/Scrapy/tutorial</span>
<span style="color: #888888;">You can start your first spider with:</span>
<span style="color: #888888;"> cd tutorial</span>
<span style="color: #888888;"> scrapy genspider example example.com</span>
</pre>
</div>
<br />
<!-- HTML generated using hilite.me --><br />
<div style="background: #ffffff; border-width: 0.1em 0.1em 0.1em 0.8em; border: solid gray; overflow: auto; padding: 0.2em 0.6em; width: auto;">
<pre style="line-height: 125%; margin: 0;"><span style="color: #c65d09; font-weight: bold;">nutt@nutt-pc:~/OneDrive/Scrapy$</span> tree tutorial/
<span style="color: #888888;">tutorial/</span>
<span style="color: #888888;">├── scrapy.cfg</span>
<span style="color: #888888;">└── tutorial</span>
<span style="color: #888888;"> ├── __init__.py</span>
<span style="color: #888888;"> ├── items.py</span>
<span style="color: #888888;"> ├── pipelines.py</span>
<span style="color: #888888;"> ├── settings.py</span>
<span style="color: #888888;"> └── spiders</span>
<span style="color: #888888;"> └── __init__.py</span>
<span style="color: #888888;">2 directories, 6 files</span>
</pre>
</div>
<br />
Create new dmoz_spider.py in spiders directory<br />
and modify items.py on top directory<br />
<br />
<!-- HTML generated using hilite.me --><br />
<div style="background: #ffffff; border-width: 0.1em 0.1em 0.1em 0.8em; border: solid gray; overflow: auto; padding: 0.2em 0.6em; width: auto;">
<pre style="line-height: 125%; margin: 0;"><span style="color: #c65d09; font-weight: bold;">nutt@nutt-pc:~/OneDrive/Scrapy/tutorial$</span> scrapy crawl dmoz
<span style="color: #888888;">2016-08-22 23:08:01 [scrapy] INFO: Scrapy 1.1.2 started (bot: tutorial)</span>
<span style="color: #888888;">2016-08-22 23:08:01 [scrapy] INFO: Overridden settings: {'NEWSPIDER_MODULE': 'tutorial.spiders', 'SPIDER_MODULES': ['tutorial.spiders'], 'ROBOTSTXT_OBEY': True, 'BOT_NAME': 'tutorial'}</span>
<span style="color: #888888;">2016-08-22 23:08:02 [scrapy] INFO: Enabled extensions:</span>
<span style="color: #888888;">['scrapy.extensions.logstats.LogStats',</span>
<span style="color: #888888;"> 'scrapy.extensions.telnet.TelnetConsole',</span>
<span style="color: #888888;"> 'scrapy.extensions.corestats.CoreStats']</span>
<span style="color: #888888;">2016-08-22 23:08:02 [scrapy] INFO: Enabled downloader middlewares:</span>
<span style="color: #888888;">['scrapy.downloadermiddlewares.robotstxt.RobotsTxtMiddleware',</span>
<span style="color: #888888;"> 'scrapy.downloadermiddlewares.httpauth.HttpAuthMiddleware',</span>
<span style="color: #888888;"> 'scrapy.downloadermiddlewares.downloadtimeout.DownloadTimeoutMiddleware',</span>
<span style="color: #888888;"> 'scrapy.downloadermiddlewares.useragent.UserAgentMiddleware',</span>
<span style="color: #888888;"> 'scrapy.downloadermiddlewares.retry.RetryMiddleware',</span>
<span style="color: #888888;"> 'scrapy.downloadermiddlewares.defaultheaders.DefaultHeadersMiddleware',</span>
<span style="color: #888888;"> 'scrapy.downloadermiddlewares.redirect.MetaRefreshMiddleware',</span>
<span style="color: #888888;"> 'scrapy.downloadermiddlewares.httpcompression.HttpCompressionMiddleware',</span>
<span style="color: #888888;"> 'scrapy.downloadermiddlewares.redirect.RedirectMiddleware',</span>
<span style="color: #888888;"> 'scrapy.downloadermiddlewares.cookies.CookiesMiddleware',</span>
<span style="color: #888888;"> 'scrapy.downloadermiddlewares.chunked.ChunkedTransferMiddleware',</span>
<span style="color: #888888;"> 'scrapy.downloadermiddlewares.stats.DownloaderStats']</span>
<span style="color: #888888;">2016-08-22 23:08:02 [scrapy] INFO: Enabled spider middlewares:</span>
<span style="color: #888888;">['scrapy.spidermiddlewares.httperror.HttpErrorMiddleware',</span>
<span style="color: #888888;"> 'scrapy.spidermiddlewares.offsite.OffsiteMiddleware',</span>
<span style="color: #888888;"> 'scrapy.spidermiddlewares.referer.RefererMiddleware',</span>
<span style="color: #888888;"> 'scrapy.spidermiddlewares.urllength.UrlLengthMiddleware',</span>
<span style="color: #888888;"> 'scrapy.spidermiddlewares.depth.DepthMiddleware']</span>
<span style="color: #888888;">2016-08-22 23:08:02 [scrapy] INFO: Enabled item pipelines:</span>
<span style="color: #888888;">[]</span>
<span style="color: #888888;">2016-08-22 23:08:02 [scrapy] INFO: Spider opened</span>
<span style="color: #888888;">2016-08-22 23:08:02 [scrapy] INFO: Crawled 0 pages (at 0 pages/min), scraped 0 items (at 0 items/min)</span>
<span style="color: #888888;">2016-08-22 23:08:02 [scrapy] DEBUG: Telnet console listening on 127.0.0.1:6023</span>
<span style="color: #888888;">2016-08-22 23:08:03 [scrapy] DEBUG: Crawled (200) <GET http://www.dmoz.org/robots.txt> (referer: None)</span>
<span style="color: #888888;">2016-08-22 23:08:04 [scrapy] DEBUG: Crawled (200) <GET http://www.dmoz.org/Computers/Programming/Languages/Python/Books/> (referer: None)</span>
<span style="color: #888888;">2016-08-22 23:08:04 [scrapy] DEBUG: Crawled (200) <GET http://www.dmoz.org/Computers/Programming/Languages/Python/Resources/> (referer: None)</span>
<span style="color: #888888;">2016-08-22 23:08:04 [scrapy] INFO: Closing spider (finished)</span>
<span style="color: #888888;">2016-08-22 23:08:04 [scrapy] INFO: Dumping Scrapy stats:</span>
<span style="color: #888888;">{'downloader/request_bytes': 734,</span>
<span style="color: #888888;"> 'downloader/request_count': 3,</span>
<span style="color: #888888;"> 'downloader/request_method_count/GET': 3,</span>
<span style="color: #888888;"> 'downloader/response_bytes': 16908,</span>
<span style="color: #888888;"> 'downloader/response_count': 3,</span>
<span style="color: #888888;"> 'downloader/response_status_count/200': 3,</span>
<span style="color: #888888;"> 'finish_reason': 'finished',</span>
<span style="color: #888888;"> 'finish_time': datetime.datetime(2016, 8, 22, 16, 8, 4, 322225),</span>
<span style="color: #888888;"> 'log_count/DEBUG': 4,</span>
<span style="color: #888888;"> 'log_count/INFO': 7,</span>
<span style="color: #888888;"> 'response_received_count': 3,</span>
<span style="color: #888888;"> 'scheduler/dequeued': 2,</span>
<span style="color: #888888;"> 'scheduler/dequeued/memory': 2,</span>
<span style="color: #888888;"> 'scheduler/enqueued': 2,</span>
<span style="color: #888888;"> 'scheduler/enqueued/memory': 2,</span>
<span style="color: #888888;"> 'start_time': datetime.datetime(2016, 8, 22, 16, 8, 2, 550009)}</span>
<span style="color: #888888;">2016-08-22 23:08:04 [scrapy] INFO: Spider closed (finished)</span>
</pre>
</div>
<br />
<!-- HTML generated using hilite.me --><br />
<div style="background: #ffffff; border-width: 0.1em 0.1em 0.1em 0.8em; border: solid gray; overflow: auto; padding: 0.2em 0.6em; width: auto;">
<pre style="line-height: 125%; margin: 0;"><span style="color: #c65d09; font-weight: bold;">nutt@nutt-pc:~/OneDrive/Scrapy$</span> tree tutorial/
<span style="color: #888888;">tutorial/</span>
<span style="color: #888888;">├── </span><span style="color: blue;">Books.html</span>
<span style="color: #888888;">├── </span><span style="color: blue;">Resources.html</span>
<span style="color: #888888;">├── scrapy.cfg</span>
<span style="color: #888888;">└── tutorial</span>
<span style="color: #888888;"> ├── __init__.py</span>
<span style="color: #888888;"> ├── __init__.pyc</span>
<span style="color: #888888;"> ├── items.py</span>
<span style="color: #888888;"> ├── pipelines.py</span>
<span style="color: #888888;"> ├── settings.py</span>
<span style="color: #888888;"> ├── settings.pyc</span>
<span style="color: #888888;"> └── spiders</span>
<span style="color: #888888;"> ├── dmoz_spider.py</span>
<span style="color: #888888;"> ├── dmoz_spider.pyc</span>
<span style="color: #888888;"> ├── __init__.py</span>
<span style="color: #888888;"> └── __init__.pyc</span>
<span style="color: #888888;">2 directories, 13 files</span>
</pre>
</div>
<br />
Have new 2 files created <span style="color: blue;">Books.html</span> and <span style="color: blue;">Resources.html </span>that are scrapy.http.Response objects file of scrapy.Request objects<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />พี่นัทhttp://www.blogger.com/profile/14291406590594822628noreply@blogger.com0tag:blogger.com,1999:blog-2488490746136830746.post-22998233841216096592016-07-15T14:28:00.001+07:002016-07-15T14:29:03.949+07:00Resize boot2docker VM Disk (Docker toolbox for Windows)<br />
<b>Stop docker machine</b><br />
<br />
<!-- HTML generated using hilite.me --><br />
<div style="background: #ffffff; border-width: 0.1em 0.1em 0.1em 0.8em; border: solid gray; overflow: auto; padding: 0.2em 0.6em; width: auto;">
<pre style="line-height: 125%; margin: 0;">C:\Program Files\Docker Toolbox>docker-machine stop
Stopping <span style="background-color: #fff0f0;">"default"</span>...
Machine <span style="background-color: #fff0f0;">"default"</span> was stopped.
</pre>
</div>
<br />
<b>Clone existing VMDK disk to VDI disk</b><br />
<!-- HTML generated using hilite.me --><br />
<div style="background: #ffffff; border-width: 0.1em 0.1em 0.1em 0.8em; border: solid gray; overflow: auto; padding: 0.2em 0.6em; width: auto;">
<pre style="line-height: 125%; margin: 0;">C:\Program Files\Oracle\VirtualBox>vboxmanage clonehd C:\Users\nutt-\.docker\machine\machines\default\disk.vmdk C:\Users\nutt-\.docker\machine\machines\default\disk.vdi --format VDI --variant Standard
<span style="color: #6600ee; font-weight: bold;">0</span>%...<span style="color: #6600ee; font-weight: bold;">10</span>%...<span style="color: #6600ee; font-weight: bold;">20</span>%...<span style="color: #6600ee; font-weight: bold;">30</span>%...<span style="color: #6600ee; font-weight: bold;">40</span>%...<span style="color: #6600ee; font-weight: bold;">50</span>%...<span style="color: #6600ee; font-weight: bold;">60</span>%...<span style="color: #6600ee; font-weight: bold;">70</span>%...<span style="color: #6600ee; font-weight: bold;">80</span>%...<span style="color: #6600ee; font-weight: bold;">90</span>%...<span style="color: #6600ee; font-weight: bold;">100</span>%
Clone medium created in format <span style="background-color: #fff0f0;">'VDI'</span>. UUID: ce<span style="color: #6600ee; font-weight: bold;">997152-32</span>d<span style="color: #6600ee; font-weight: bold;">1-442</span>e-b<span style="color: #6600ee; font-weight: bold;">3</span>e<span style="color: #6600ee; font-weight: bold;">2-34</span>cdc<span style="color: #6600ee; font-weight: bold;">2</span>d<span style="color: #6600ee; font-weight: bold;">74</span>d<span style="color: #6600ee; font-weight: bold;">6</span>b
</pre>
</div>
<br />
<b>Resize VDI disk to desired size</b><br />
<br />
<!-- HTML generated using hilite.me --><br />
<div style="background: #ffffff; border-width: 0.1em 0.1em 0.1em 0.8em; border: solid gray; overflow: auto; padding: 0.2em 0.6em; width: auto;">
<pre style="line-height: 125%; margin: 0;">C:\Program Files\Oracle\VirtualBox>vboxmanage modifyhd C:\Users\nutt-\.docker\machine\machines\default\disk.vdi --resize <span style="color: #6600ee; font-weight: bold;">102400</span>
<span style="color: #6600ee; font-weight: bold;">0</span>%...<span style="color: #6600ee; font-weight: bold;">10</span>%...<span style="color: #6600ee; font-weight: bold;">20</span>%...<span style="color: #6600ee; font-weight: bold;">30</span>%...<span style="color: #6600ee; font-weight: bold;">40</span>%...<span style="color: #6600ee; font-weight: bold;">50</span>%...<span style="color: #6600ee; font-weight: bold;">60</span>%...<span style="color: #6600ee; font-weight: bold;">70</span>%...<span style="color: #6600ee; font-weight: bold;">80</span>%...<span style="color: #6600ee; font-weight: bold;">90</span>%...<span style="color: #6600ee; font-weight: bold;">100</span>%
</pre>
</div>
<br />
<br />
<b>Download </b><a href="https://sourceforge.net/projects/gparted/?source=typ_redirect" target="_blank">GParted </a>partition manager or use Ubuntu bootable setup CD<br />
<br />
<b>Add new ISO in VirtualBox as below</b><br />
<br />
<i><b>GParted live</b></i><br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj_7KWXmbK7dSJAaVPBaIS-nvdI0UIT6ypRLynlgpNMH02n7SArIj5j2fnPn3huzvqs2pUfO4Exc2GJFiPDlX5o_ppV0hMWERaiNI9DL6fp103hoOQbQCc9Qh79brbH9F223ctAm0ctcWYh/s1600/Capture.PNG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="422" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj_7KWXmbK7dSJAaVPBaIS-nvdI0UIT6ypRLynlgpNMH02n7SArIj5j2fnPn3huzvqs2pUfO4Exc2GJFiPDlX5o_ppV0hMWERaiNI9DL6fp103hoOQbQCc9Qh79brbH9F223ctAm0ctcWYh/s640/Capture.PNG" width="640" /></a></div>
<br />
<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjDIiA8UQqWGD41XUqkSaFO9Z-aUmBZLLehkqJglwGB3pdqA1bPdGFOxthQNobayZ5Ph2XHdRzVMDxpH2HPHDJQi6L5S8RpdgNQGGtsT-H61dMiHcSlAJd2adjylH0K5yGM_X_lbeZ_xMFw/s1600/VirtualBox_default_15_07_2016_08_17_56.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="480" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjDIiA8UQqWGD41XUqkSaFO9Z-aUmBZLLehkqJglwGB3pdqA1bPdGFOxthQNobayZ5Ph2XHdRzVMDxpH2HPHDJQi6L5S8RpdgNQGGtsT-H61dMiHcSlAJd2adjylH0K5yGM_X_lbeZ_xMFw/s640/VirtualBox_default_15_07_2016_08_17_56.png" width="640" /></a></div>
<br />
<br />
<b><i>Ubuntu bootable CD</i></b><br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg1ZGnWgBiUzi9OTeQdfjf5uMtLpWgAdJGWZrsdJl7-R6ofwvpLM8AV_nRdXf0q6WZR-VBV093XXVzeSMEVqt9jrsEZaXB1D3XgV-c4A0y1-JHMJhFm584qAdsn_7TUSsR1MXGWmIvaucP2/s1600/VirtualBox_default_15_07_2016_12_23_53.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="480" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEg1ZGnWgBiUzi9OTeQdfjf5uMtLpWgAdJGWZrsdJl7-R6ofwvpLM8AV_nRdXf0q6WZR-VBV093XXVzeSMEVqt9jrsEZaXB1D3XgV-c4A0y1-JHMJhFm584qAdsn_7TUSsR1MXGWmIvaucP2/s640/VirtualBox_default_15_07_2016_12_23_53.png" width="640" /></a></div>
<br />
Both method have the same result. Now your docker machine reside on 100GB VDI disk.<br />
<br />
<br />
<br />
<br />
<br />
<br />พี่นัทhttp://www.blogger.com/profile/14291406590594822628noreply@blogger.com1tag:blogger.com,1999:blog-2488490746136830746.post-3759021332548176472016-07-11T10:51:00.000+07:002016-07-11T11:28:49.696+07:00Reformat delimiter-based to fixed-length file<br />
<br />
<br />
<br />
Project source at <a href="https://github.com/nutthaphon/reformat" target="_blank">Github</a><br />
<br />
<br />
<br />
<br />พี่นัทhttp://www.blogger.com/profile/14291406590594822628noreply@blogger.com0