yum clean metadata
yum install httpd
yum install firefox
After I added a bridged network, here is a screenshot that proves that it is working.
First, after I started the httpd server, I can see the apache server default page from firefox inside the CentOS vm.
Second, after turned connected to the bridged network, I can open the server from my local Windows OS. I have highlighted the ip address and MAC of the network device.
I am trying to set up a Django application running in production where the OS is Redhat. Before pissing off the system admin, I need to test all the possible catastrophic tweaks on a sandbox. You can either start a Redhat virtualbox on the cloud like AWS or you can maybe run a CentOS for your local virtualbox, which is what I am going to try.
If you go the CentOS website, you will realize the existences of tons of different downloadable images.
Here is a very good explanation of what are those images for, I am planning to start with the minimal image which is 395MB about 10% of the size of CentOS DVD 1 and 2 :).
The installation is pretty straight forward, you need to mount the ISO to the virtualbox and make sure you boot from CD first instead of harddrive, then just follow the installation.
In the end, you probably have to log in as user “root” and then you had better create a new user by adduser and passwd, otherwise, CentOS won’t be able allowing root to even start a GUI.
After I am done with the installation, the network had a hiccup which I resolved thanks to this post. I just need to change the nm_manager from yes to no and then yum will work properly.
After ‘yum groupinstall desktop’, you will be able to see a pretty ghetto desktop that doesn’t have a high enough resolution to fit your 24+ in monitor.
Next, you need to ‘insert guest additions’ and then the resolution will change to whatever resolution as you scale your screen.
However, the last thing to worry about is the text in terminal is a bit f* up, I posted a question on stackexchange and fingers crossed someone gonna answer it.
The total installation including goolging and debugging took my girl friend about half an hour with my help.
I was looking at the settings.py file of a Django project and it turned out there is a line of code likes this:
BASE_DIR = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
It basically created variable called BASE_DIR pointing to the project directory.
__file__: the name of the running script, won’t exist in shell
os.path.abspath: the absolute path of the given file by joining current working directory and file name
os.path.dirname: the directory name of the absolute path, = os.path.split(path)[0]
The os.path.join is pretty intelligent:
“Join one or more path components intelligently. The return value is the concatenation of pathand any members of *paths with exactly one directory separator (os.sep) following each non-empty part except the last, meaning that the result will only end in a separator if the last part is empty. If a component is an absolute path, all previous components are thrown away and joining continues from the absolute path component.”
>>> os.path.join(‘Users’, ‘a59347’, ‘Desktop’)
‘Users/a59347/Desktop’
>>> os.path.join(‘Users’, ‘/Users/a59347’, ‘Desktop’)
‘/Users/a59347/Desktop’
>>> os.path.join(‘Users’, ‘/Users1/a59347’, ‘Desktop’)
‘/Users1/a59347/Desktop’
This is a great Python Django tutorial, however, the Django version the author was using might be slightly out of date.
Keep that in mind while following his tutorial.
Microsoft ML API Design
There are a lot of work you can do inside the HBase shell. You can list the tables, you can get and put records ..etc.
**************** DESCRIBE ****************
hbase(main):003:0> describe ‘a59347_tco’
Table mykickasstable is ENABLED
mykickasstable
COLUMN FAMILIES DESCRIPTION
{
NAME => ‘OTHER’,
DATA_BLOCK_ENCODING => ‘NONE’,
BLOOMFILTER => ‘NONE’,
REPLICATION_SCOPE => ‘0’,
VERSIONS => ‘3’,
COMPRESSION => ‘NONE’,
MIN_VERSIONS => ‘0’,
TTL => ‘FOREVER’,
KEEP_DELETED_CELLS => ‘FALSE’,
BLOCKSIZE => ‘65536’,
IN_MEMORY => ‘false’,
BLOCKCACHE => ‘false’
}
…
TTL is short for Time To Live, `FOREVER` means the data you put in will never expire. It will be a great functionality if you have some use cases where always want to keep a certain amount of data like ‘only store 1 year of data’. In that case, you can probably set the TTL to be one year and it will automatically delete the records after it expired.
BLOCKSIZE is 64MB as the default blocksize for HDFS.
You can also use status command to check the running condition of your hBase cluster, it will return something like this:
hbase(main):016:0> status ‘simple’
8 live servers
server16.datafireball.com:60020 1434485581215
requestsPerSecond=0.0,
numberOfOnlineRegions=4,
usedHeapMB=346,
maxHeapMB=1583,
numberOfStores=7,
numberOfStorefiles=9,
storefileUncompressedSizeMB=12530,
storefileSizeMB=12535,
compressionRatio=1.0004,
memstoreSizeMB=0,
storefileIndexSizeMB=0,
readRequestsCount=14382,
writeRequestsCount=0,
rootIndexSizeKB=126,
totalStaticIndexSizeKB=12657,
totalStaticBloomSizeKB=13420,
totalCompactingKVs=0,
currentCompactedKVs=0,
compactionProgressPct=NaN,
coprocessors=[]
We can see there are plenty of parameters you can refer to help you understanding the running condition of your cluster, understand what they mean will be a long process but super helpful as a big data system admin.
Here is a great blog that shows some practical business using Python.
This post is about how to use Python to read from Google Form where I can image it could be a good resource for crowd sourcing.
Link here
datafireball 