As the scale of web applications increases, performance optimization considerations are more frequently included in initial design. One optimization technique used extensively is caching. A cache contains pre-processed data that is ready to use without redoing the processing. Processing may include extensive computation and accessing data over a network or on disk. Keep the details out of your code One design consideration when introducing a caching mechanism in your code is to keep the details out of your code. Most caches are just simple key value stores, and so it’s tempting to introduce......
Continue Reading
I’ve had several conversations recently about caching as it relates to big data. As a result of these discussions I wanted to review some details that should be considered when deciding if a cache is necessary and how to cache big data when it is necessary. What is a Cache? The purpose of a cache is to duplicate frequently accessed or important data in such a way that it can be accessed very fast and close to where it is needed. Caching generally moves data from a low cost, high density location (e.g.......
Continue Reading
Another tool for monitoring the performance and health of a MongoDB node is mongostat. You’ll recall that mongotop shows the time in milliseconds that a mongo node spent accessing (read and write) a particular collection. mongostat on the other hand provides more detailed information about the state of a mongo node, including disk usage, data throughput, index misses, locks, etc. However, the data is general to the mongo node and doesn’t indicate which database or collection the status refers to. As you would expect, both utilities, mongotop and mongostat, are required to get......
Continue Reading
In the process of tuning the performance of a MongoDB replica set, it’s useful to be able to observe mongod directly, as opposed to inferring what it’s doing by watching the output of top, for example. For that reason MongoDB comes with a utility, mongotop. The output of mongotop indicates the amount of time the mongod process spend reading and writing to a specific collection during the update interval. I used the following command to run mongotop on an authentication enabled replica set with a two second interval. [watrous@d1t0156g ~]# mongotop -p -u......
Continue Reading
MongoDB connections accommodate a ReadPreference, which in a clustered environment, like a replicaset, indicates how to select the best host for a query. One major consideration when setting the read preference is whether or not you can live with eventually consistent reads, since SECONDARY hosts may lag behind the PRIMARY. Some of the options you can choose include: PRIMARY: This will ensure the most consistency, but also concentrates all your queries on a single host. SECONDARY: This will distribute your queries among secondary nodes and may lag in consistency with the primary primaryPreferred:......
Continue Reading