I have been trying to tune my MysSQL databases, but always having a very low cache hit rate.
I suspect partially because I run a sitemap crawler that is crawling 1.3 million pages. But, besides that, the most site traffic is on a dozen of those pages.
While I have a large table_open_cache, only a small number of tables ever seem to be open at any time.
my my.cnf
[mysqld]
sort_buffer_size = 500k
read_rnd_buffer_size = 500k
tmp_table_size = 4G
max_heap_table_size = 4G
max_connections = 100
wait_timeout = 20000
interactive_timeout = 20000
key_buffer =2G
max_allowed_packet=268435456
thread_cache_size = 8
table_open_cache = 42997
join_buffer_size = 2M
table_definition_cache = 46258
low_priority_updates=1
concurrent_insert=ALWAYS
query_cache_type=1
# * Query Cache Configuration
#
query_cache_limit = 256k
query_cache_size = 256M
#tmp_table_size = 2048M
#max_heap_table_size = 2048M
innodb_buffer_pool_size = 9096M
innodb_flush_method = O_DIRECT
default-storage-engine=MyISAM
innodb_file_per_table=1
open_files_limit=85594
note the temp table lines being commented out was just something i wanted to try
the results of the tuning scripts are
# perl mysqltuner.pl
>> MySQLTuner 1.4.0 - Major Hayden <major@mhtx.net>
>> Bug reports, feature requests, and downloads at
>> Run with '--help' for additional options and output filtering
[OK] Currently running supported MySQL version 5.6.23
[OK] Operating on 64-bit architecture
-------- Storage Engine Statistics -------------------------------------------
[--] Status: +ARCHIVE +BLACKHOLE +CSV -FEDERATED +InnoDB +MRG_MYISAM
[--] Data in MyISAM tables: 2G (Tables: 2308)
[--] Data in InnoDB tables: 1G (Tables: 385)
[--] Data in PERFORMANCE_SCHEMA tables: 0B (Tables: 52)
[!!] Total fragmented tables: 19
-------- Security Recommendations -------------------------------------------
[OK] All database users have passwords assigned
-------- Performance Metrics -------------------------------------------------
[--] Up for: 1d 8h 44m 14s (23M q [202.135 qps], 1M conn, TX: 110B, RX: 157B)
[--] Reads / Writes: 70% / 30%
[--] Total buffers: 15.1G global + 3.4M per thread (100 max threads)
[OK] Maximum possible memory usage: 15.5G (49% of installed RAM)
[OK] Slow queries: 0% (36/23M)
[OK] Highest usage of available connections: 64% (64/100)
[OK] Key buffer size / total MyISAM indexes: 2.0G/287.1M
[OK] Key buffer hit rate: 98.1% (544M cached / 10M reads)
[OK] Query cache efficiency: 54.9% (9M cached / 16M selects)
[OK] Query cache prunes per day: 0
[OK] Sorts requiring temporary tables: 4% (66K temp sorts / 1M sorts)
[!!] Joins performed without indexes: 1110
[OK] Temporary tables created on disk: 19% (254K on disk / 1M total)
[OK] Thread cache hit rate: 99% (3K created / 1M connections)
[!!] Table cache hit rate: 0% (924 open / 116K opened)
[OK] Open file limit used: 0% (538/86K)
[OK] Table locks acquired immediately: 99% (15M immediate / 16M locks)
[OK] InnoDB buffer pool / data size: 8.9G/1.1G
[OK] InnoDB log waits: 0
-------- Recommendations -----------------------------------------------------
General recommendations:
Run OPTIMIZE TABLE to defragment tables for better performance
Adjust your join queries to always utilize indexes
Increase table_open_cache gradually to avoid file descriptor limits
Read this before increasing table_open_cache over 64:
Variables to adjust:
join_buffer_size (> 2.0M, or always use indexes with joins)
table_open_cache (> 42997)
[root@server ~]# pico /etc/my.cmf
[root@server ~]# pico /etc/my.cnf
[root@server ~]# perl mysqltuner.pl
>> MySQLTuner 1.4.0 - Major Hayden <major@mhtx.net>
>> Bug reports, feature requests, and downloads at
>> Run with '--help' for additional options and output filtering
[OK] Currently running supported MySQL version 5.6.23
[OK] Operating on 64-bit architecture
-------- Storage Engine Statistics -------------------------------------------
[--] Status: +ARCHIVE +BLACKHOLE +CSV -FEDERATED +InnoDB +MRG_MYISAM
[--] Data in MyISAM tables: 2G (Tables: 2308)
[--] Data in InnoDB tables: 1G (Tables: 385)
[--] Data in PERFORMANCE_SCHEMA tables: 0B (Tables: 52)
[!!] Total fragmented tables: 26
-------- Security Recommendations -------------------------------------------
[OK] All database users have passwords assigned
-------- Performance Metrics -------------------------------------------------
[--] Up for: 1d 9h 6m 22s (24M q [204.238 qps], 1M conn, TX: 113B, RX: 161B)
[--] Reads / Writes: 70% / 30%
[--] Total buffers: 15.1G global + 3.4M per thread (100 max threads)
[OK] Maximum possible memory usage: 15.5G (49% of installed RAM)
[OK] Slow queries: 0% (36/24M)
[OK] Highest usage of available connections: 64% (64/100)
[OK] Key buffer size / total MyISAM indexes: 2.0G/288.8M
[OK] Key buffer hit rate: 98.1% (554M cached / 10M reads)
[OK] Query cache efficiency: 54.8% (9M cached / 17M selects)
[OK] Query cache prunes per day: 0
[OK] Sorts requiring temporary tables: 4% (67K temp sorts / 1M sorts)
[!!] Joins performed without indexes: 1144
[OK] Temporary tables created on disk: 19% (260K on disk / 1M total)
[OK] Thread cache hit rate: 99% (3K created / 1M connections)
[!!] Table cache hit rate: 2% (3K open / 118K opened)
[OK] Open file limit used: 5% (4K/86K)
[OK] Table locks acquired immediately: 99% (16M immediate / 16M locks)
[OK] InnoDB buffer pool / data size: 8.9G/1.1G
[OK] InnoDB log waits: 0
-------- Recommendations -----------------------------------------------------
General recommendations:
Run OPTIMIZE TABLE to defragment tables for better performance
Adjust your join queries to always utilize indexes
Increase table_open_cache gradually to avoid file descriptor limits
Read this before increasing table_open_cache over 64:
Variables to adjust:
join_buffer_size (> 2.0M, or always use indexes with joins)
table_open_cache (> 42997)
and
# sh tuning-primer.sh
-- MYSQL PERFORMANCE TUNING PRIMER --
- By: Matthew Montgomery -
MySQL Version 5.6.23 x86_64
Uptime = 1 days 9 hrs 10 min 20 sec
Avg. qps = 204
Total Questions = 24417051
Threads Connected = 2
Warning: Server has not been running for at least 48hrs.
It may not be safe to use these recommendations
To find out more information on how each of these
runtime variables effects performance visit:
Visit
for info about MySQL's Enterprise Monitoring and Advisory Service
SLOW QUERIES
The slow query log is NOT enabled.
Current long_query_time = 10.000000 sec.
You have 36 out of 24417072 that take longer than 10.000000 sec. to complete
Your long_query_time seems to be fine
BINARY UPDATE LOG
The binary update log is NOT enabled.
You will not be able to do point in time recovery
See
WORKER THREADS
Current thread_cache_size = 8
Current threads_cached = 6
Current threads_per_sec = 0
Historic threads_per_sec = 0
Your thread_cache_size is fine
MAX CONNECTIONS
Current max_connections = 100
Current threads_connected = 3
Historic max_used_connections = 64
The number of used connections is 64% of the configured maximum.
Your max_connections variable seems to be fine.
INNODB STATUS
Current InnoDB index space = 898 M
Current InnoDB data space = 1.07 G
Current InnoDB buffer pool free = 1 %
Current innodb_buffer_pool_size = 8.88 G
Depending on how much space your innodb indexes take up it may be safe
to increase this value to up to 2 / 3 of total system memory
MEMORY USAGE
Max Memory Ever Allocated : 11.35 G
Configured Max Per-thread Buffers : 335 M
Configured Max Global Buffers : 11.14 G
Configured Max Memory Limit : 11.47 G
Physical Memory : 31.31 G
Max memory limit seem to be within acceptable norms
KEY BUFFER
Current MyISAM index space = 289 M
Current key_buffer_size = 2.00 G
Key cache miss rate is 1 : 52
Key buffer free ratio = 80 %
Your key_buffer_size seems to be fine
QUERY CACHE
Query cache is enabled
Current query_cache_size = 256 M
Current query_cache_used = 67 M
Current query_cache_limit = 256 K
Current Query cache Memory fill ratio = 26.39 %
Current query_cache_min_res_unit = 4 K
MySQL won't cache query results that are larger than query_cache_limit in size
SORT OPERATIONS
Current sort_buffer_size = 500 K
Current read_rnd_buffer_size = 500 K
Sort buffer seems to be fine
JOINS
Current join_buffer_size = 2.00 M
You have had 1162 queries where a join could not use an index properly
You should enable "log-queries-not-using-indexes"
Then look for non indexed joins in the slow query log.
If you are unable to optimize your queries you may want to increase your
join_buffer_size to accommodate larger joins in one pass.
Note! This script will still suggest raising the join_buffer_size when
ANY joins not using indexes are found.
OPEN FILES LIMIT
Current open_files_limit = 86104 files
The open_files_limit should typically be set to at least 2x-3x
that of table_cache if you have heavy MyISAM usage.
Your open_files_limit value seems to be fine
TABLE CACHE
Current table_open_cache = 42997 tables
Current table_definition_cache = 46258 tables
You have a total of 2774 tables
You have 3029 open tables.
The table_cache value seems to be fine
TEMP TABLES
Current max_heap_table_size = 4.00 G
Current tmp_table_size = 4.00 G
Of 1337828 temp tables, 16% were created on disk
Created disk tmp tables ratio seems fine
TABLE SCANS
Current read_buffer_size = 128 K
Current table scan ratio = 190 : 1
read_buffer_size seems to be fine
TABLE LOCKING
Current Lock Wait ratio = 1 : 601
You may benefit from selective use of InnoDB.
I do run a auto repair and optimize as well as a flush query cache cron every hour.
edit: i have noticed certain behavior, after restarting mysql the tale_open_cache reads hit rate 99% xxx open / xxx opened up till 2k open / 2k opened then the opened goes up and up while open drops down to between 680 and 980..ish it now says 908 open 28k opened seemsto get to that 2k then closes most of them
