Monday, August 30, 2021
An Interesting AWK Script I wrote last night
Of course it’s for working with Chia!!! :)
HVAR=”Count by time taken”
HEADER
$GREP -i qualities $LONGPLOTS | $AWK {’print $11′} | $AWK -F. {’print $1′} | $SORT -n | $UNIQ -c |
$AWK -v PER=”%” -v LONGP=”$LONGP” -v LOGELIT=”$LOGELIT”
‘{ printf(”%1g Sec - %1g CNT - %1.2g %0s Longplots %.2g %0s Totaln”, $2, $1, ($1 / LONGP * 100), PER, ($1 / LOGELIT * 100), PER) }’
RESULTS
Mon 30 Aug 2021 04:41:01 AM MST —————————————————————- – Count by time taken —————————————————————- 5 Sec - 750 CNT - 33 % Longplots 1.1 % Total 6 Sec - 523 CNT - 23 % Longplots 0.76 % Total 7 Sec - 135 CNT - 5.9 % Longplots 0.2 % Total 8 Sec - 93 CNT - 4.1 % Longplots 0.14 % Total 9 Sec - 42 CNT - 1.8 % Longplots 0.061 % Total 10 Sec - 46 CNT - 2 % Longplots 0.067 % Total 11 Sec - 138 CNT - 6.1 % Longplots 0.2 % Total
The awk script is basically parsing a temp file I created out of the log files
LONGPLOTS =/tmp/longplots.tmp
root@ms2:~/bin# grep qualities /tmp/longplots.tmp | tail -10 2021-08-23T07:02:09.920 harvester chia.harvester.harvester: WARNING Looking up qualities on /nas/h/plots/plot-k32-2021-06-01-02-23-8bbea0477b514310c244984ceef47b0f41e79e0dd3cf6b716fb1017d731ecf55.plot took: 21.72266125679016. This should be below 5 seconds to minimize risk of losing rewards. 2021-08-23T07:02:09.920 harvester chia.harvester.harvester: WARNING Looking up qualities on /nas/n1b/plots/plot-k32-2021-04-30-15-27-045de6d0fe1f16b3c478ac43575bd37c4d4a3b669aa00da6b8939b36935edf73.plot took: 21.722743272781372. This should be below 5 seconds to minimize risk of losing rewards. 2021-08-23T07:02:10.097 harvester chia.harvester.harvester: WARNING Looking up qualities on /nas/a/plots/plot-k32-2021-07-04-22-22-fa73ce855e1db23a577eb4be390581e324b4a64174ae4a692d5ce98b1cc96a5b.plot took: 12.023487567901611. This should be below 5 seconds to minimize risk of losing rewards. 2021-08-23T07:02:10.097 harvester chia.harvester.harvester: WARNING Looking up qualities on /nas/g/plots/plot-k32-2021-06-28-21-58-7f57303d0b82aefe6245b09517f6a9de8e92179c358c4596e0585882ed36997a.plot took: 12.023878574371338. This should be below 5 seconds to minimize risk of losing rewards. 2021-08-23T07:03:31.133 harvester chia.harvester.harvester: WARNING Looking up qualities on /nas/d/plots/plot-k32-2021-07-30-06-16-1ea66b80292c284646673762cfcb12a947835144163f311714d1dd9c8e7f97b2.plot took: 10.404966831207275. This should be below 5 seconds to minimize risk of losing rewards. 2021-08-23T07:05:13.834 harvester chia.harvester.harvester: WARNING Looking up qualities on /nas/d/plots/plot-k32-2021-07-27-22-53-c8272e00e88c8335f66e938c8341da0d945cb616dcbb5d676fd85e071589bf51.plot took: 8.03150486946106. This should be below 5 seconds to minimize risk of losing rewards. 2021-08-23T07:24:26.028 harvester chia.harvester.harvester: WARNING Looking up qualities on /nas/d/plots/plot-k32-2021-07-24-18-17-f3a865a801ca78540713d9262e581a5a8345ebf6a107fc35844fa9c9996ff13c.plot took: 5.538493394851685. This should be below 5 seconds to minimize risk of losing rewards. 2021-08-23T07:37:34.878 harvester chia.harvester.harvester: WARNING Looking up qualities on /nas/d/plots/plot-k32-2021-07-30-06-16-1ea66b80292c284646673762cfcb12a947835144163f311714d1dd9c8e7f97b2.plot took: 5.647717714309692. This should be below 5 seconds to minimize risk of losing rewards. 2021-08-23T07:57:19.040 harvester chia.harvester.harvester: WARNING Looking up qualities on /nas/d/plots/plot-k32-2021-08-02-22-02-933a77c8d839b0870b4c4611ec4d63158c4a46cb6c95eb4d120d61ec212ed110.plot took: 5.058545112609863. This should be below 5 seconds to minimize risk of losing rewards. 2021-08-23T08:24:42.001 harvester chia.harvester.harvester: WARNING Looking up qualities on /nas/d/plots/plot-k32-2021-08-04-11-08-6b21e8bb8ef0238ec8e6d0b04e22dc0f9b91fe80c21444406cedb1c8f70ad69f.plot took: 6.0560503005981445. This should be below 5 seconds to minimize risk of losing rewards.
Grabbing the 11th field with the first awk pass (The # of seconds for the plot confirmation)
(Grabbgint the Most significant digits of the 11th field. Which is # Seconds without decimal point) with the second awk.
Sorting that by the number of seconds taken.
uniq that list with counts for number of each second. (So I have a count of each # of seconds)
and then the final awk which is the interesting piece….
$AWK -v PER=”%” -v LONGP=”$LONGP” -v LOGELIT=”$LOGELIT”
‘{ printf(”%1g Sec - %1g CNT - %1.2g %0s Longplots %.2g %0s Totaln”, $2, $1, ($1 / LONGP * 100), PER, ($1 / LOGELIT * 100), PER) }’
In order for awk to use variables. I have to pass them to awk
LONGP is the Total number of plots that took over 5 seconds
LOGELIT is the Total number of plots.
PER is a percent sign. printf uses % as a secial character and I want to use it in my output….
Then I do a printf of the output I’d like to see followed by the list of variables.
printf statement
printf(”%1g Sec - %1g CNT - %1.2g %0s Longplots %.2g %0s Totaln”,
VARIABLE LIST
$2, $1, ($1 / LONGP * 100), PER, ($1 / LOGELIT * 100), PER)
Where $1 is the count of seconds
Where $2 is the Seconds
Where PER is a percent sign “%”
Where ($1 / LONGP * 100) is the percent of that second divided by # long plots
Where ($1 / LOGELIT *100) is the percent of that second dived by Total # plots
Simple really…. ;)
—————————————————————- – Chia Debug Plots taking too long all log files —————————————————————- LONGPLOTS = 2278 TOTAL Plots = 68662 Mon 30 Aug 2021 04:41:01 AM MST —————————————————————- – Count by time taken —————————————————————- 5 Sec - 750 CNT - 33 % Longplots 1.1 % Total 6 Sec - 523 CNT - 23 % Longplots 0.76 % Total 7 Sec - 135 CNT - 5.9 % Longplots 0.2 % Total 8 Sec - 93 CNT - 4.1 % Longplots 0.14 % Total 9 Sec - 42 CNT - 1.8 % Longplots 0.061 % Total 10 Sec - 46 CNT - 2 % Longplots 0.067 % Total
Full code / results below
root@ms2:~/bin# cat chia.status.sh
#!/bin/bash
#
# Start Chia Environment
#
source /root/chia-blockchain/activate
#
# Progs I USe
#
ECHO=”/usr/bin/echo -e” # -e interupt escaped chars
CAT=/usr/bin/cat
CHIA=/root/chia-blockchain/venv/bin/chia
WC=/usr/bin/wc
LS=/usr/bin/ls
TAIL=/usr/bin/tail
HEAD=/usr/bin/head
SORT=/usr/bin/sort
TAIL=/usr/bin/tail
DATE=`/usr/bin/date`
DU=/usr/bin/du
DF=/usr/bin/df
PS=/usr/bin/ps
DMESG=/usr/bin/dmesg
AWK=/usr/bin/awk
UNIQ=/usr/bin/uniq
PLOTTIME=/root/bin/plottime
STRING=Total
GREP=/usr/bin/grep
PS=/usr/bin/ps
#
# Variables
#
HOWMANY=10 # How many ports of phase information to report on each drive
DRIVES=25 # How many drives to report free space on
LOGL=10 # How much of each log file to see
LOGH=100 # How many from the Header for Long lookup times
PLOTDRIVES=”n1a n1b n2a n2b n4a n4b sc1a sc1b sc1c sc1d sc1e zero zero2 a b c d e f g h i j k”
LOCALDRIVES=”a b c d e f”
LOGFILES=”1 2 3 4 5 6 7 8 9 10”
HEADER ()
{
$ECHO “”
$ECHO $DATE
$ECHO “—————————————————————-”
$ECHO “– “$HVAR
$ECHO “—————————————————————-”
$ECHO “”
}
DIVIDER ()
{
$ECHO “”
$ECHO “
”
$ECHO “
”
$ECHO “”
}
#
# TEMP Files / log files
#
LOGFILE=/root/.chia/mainnet/log/debug.log
LOGDIR=/root/.chia/mainnet/log
LONGPLOTS=/tmp/longplots.tmp
CHIACONS=/tmp/chiacons.tmp
#
$ECHO “”
#######################################################
HVAR=”Chia Wallet Show”
HEADER
$CHIA wallet show | $GREP -v fingerprint | $GREP -v height
#######################################################
HVAR=”Chia farm summary”
HEADER
$CHIA farm summary
#########################################################
HVAR=”Chia Show Network Connections”
HEADER
$CHIA show -c > $CHIACONS
$ECHO “FULL_NODE Connextions”
$GREP FULL $CHIACONS | $WC -l
$ECHO “”
$ECHO “WALLET Connextions”
$GREP WALLET $CHIACONS | $WC -l
DIVIDER
#######################################################
HVAR=”Drive Temp”
HEADER
$TAIL -11 /var/log/drivetemp.log | $SORT
########################################################
HVAR=”Plot Details”
HEADER
for CNT in $PLOTDRIVES
do
$DU -h /nas/$CNT/plots
if [ $CNT == “n1b” ]; then $ECHO “”; fi
if [ $CNT == “n2b” ]; then $ECHO “”; fi
if [ $CNT == “n3e” ]; then $ECHO “”; fi
if [ $CNT == “n4b” ]; then $ECHO “”; fi
if [ $CNT == “sc1e” ]; then $ECHO “”; fi
if [ $CNT == “zero2” ]; then $ECHO “”; fi
done
########################################################
HVAR=”Drive Free”
HEADER
$TAIL -$DRIVES /var/log/drivefree.log | $SORT
$ECHO “”
$DF /dev/sdc2 | $GREP dev
#########################################################
HVAR=”Memory Information”
HEADER
$CAT /proc/meminfo | $HEAD -n $HOWMANY
#########################################################
HVAR=”DMESG | Tail - $HOWMANY”
HEADER
$DMESG | $TAIL -n $HOWMANY
#########################################################
HVAR=”PSCPU10 | HEAD - 10”
HEADER
$PS -auxf | $SORT -nr -k 3 | $HEAD -n 10
#########################################################
HVAR=”Chia show PLOTing Processeses”
HEADER
$PS -elf | $GREP blockchain | $GREP -v grep
DIVIDER
#########################################################
HVAR=”Log Eligible & How Long”
HEADER
LOGELIT=`$GREP plots were $LOGFILE | $WC -l `
LONGPLOT=`$GREP losing $LOGFILE | $GREP qualities > $LONGPLOTS | $WC -l $LONGPLOTS| $AWK ‘{print $1}’ `
$LS -ltr $LOGDIR | $SORT +8
echo “——————————————————————-”
$ECHO “debug 0: ” $LOGELIT
for CNT in $LOGFILES
do
LOGELI=`$GREP plots were $LOGFILE.$CNT | $WC -l`
LONGPLOT=`$GREP losing $LOGFILE.$CNT >> $LONGPLOTS | $WC -l $LONGPLOTS | $AWK ‘{print $1}’`
$ECHO “debug $CNT: “$LOGELI
LOGELIT=$(($LOGELIT + $LOGELI))
LONGPLOTT=$(($LONGPLOTT + $LONGPLOT))
done
########################################################
HVAR=”PLOT TIMES”
HEADER
$PLOTTIME
########################################################
HVAR=” PLOTs PER DAY”
HEADER
$CAT /var/log/plots.per.day
########################################################
HVAR=”Plot Total Times [a-h]”
HEADER
$GREP “Total time” /var/log/plot*
$ECHO “”
$ECHO “
”
########################################################
HVAR=”Plot Process per drive”
HEADER
for CNT in $LOCALDRIVES
do
$ECHO $DATE “Drive “$CNT”: Last:”$HOWMANY
$ECHO “——————————————————————”
$GREP phase /var/log/plot$CNT.log | $TAIL -$HOWMANY
$ECHO “”
$ECHO “”
done
DIVIDER
########################################################
HVAR=” CHIA Debug Log Warning / Error (LAST $LOGL)”
HEADER
$ECHO `$LS -l $LOGFILE`
$ECHO “——————————————————————”
$ECHO “FOUND Proofs :”; $GREP Found 1 proof $LOGFILE
$ECHO “”
$GREP ERROR $LOGFILE |$GREP -v peername | $GREP -v Failed to fetch | $GREP -vi closing connection |$GREP -v Traceback |$TAIL -40
$ECHO “”
for CNT in $LOGFILES
do
HVAR=”CHIA Debug Log ( $CNT ) Warning / Error LAST $LOGL ”
HEADER
$ECHO `$LS -l $LOGFILE.$CNT`
$ECHO “——————————————————————”
$ECHO “FOUND Proofs :”; $GREP Found 1 proof $LOGFILE.$CNT
$ECHO “”
$GREP ERROR $LOGFILE.$CNT | $GREP -v peername | $GREP -v Failed to fetch | $GREP -vi closing connection | $GREP -v Traceback |$TAIL -$LOGL
$ECHO “”
done
########################################################
HVAR=”Total LOG Eligible Last $CNT log files = $LOGELIT”
HEADER
#######################################################
HVAR=”Chia Debug Plots taking too long all log files”
HEADER
LONGP=`$GREP qualities $LONGPLOTS | $WC -l`
$ECHO LONGPLOTS = $LONGP
$ECHO TOTAL Plots = $LOGELIT
HVAR=”Count by time taken”
HEADER
$GREP -i qualities $LONGPLOTS | $AWK {’print $11′} | $AWK -F. {’print $1′} | $SORT -n | $UNIQ -c |
$AWK -v PER=”%” -v LONGP=”$LONGP” -v LOGELIT=”$LOGELIT”
‘{ printf(”%1g Sec - %1g CNT - %1.2g %0s Longplots %.2g %0s Totaln”, $2, $1, ($1 / LONGP * 100), PER, ($1 / LOGELIT * 100), PER) }’
$ECHO “”
Mon 30 Aug 2021 04:41:01 AM MST —————————————————————- – Chia Wallet Show —————————————————————- Sync status: Synced Wallet ID 1 type STANDARD_WALLET Chia Wallet -Total Balance: 0.0 xch (0 mojo) -Pending Total Balance: 0.0 xch (0 mojo) -Spendable: 0.0 xch (0 mojo) Mon 30 Aug 2021 04:41:01 AM MST —————————————————————- – Chia farm summary —————————————————————- Farming status: Farming Total chia farmed: 0.0 User transaction fees: 0.0 Block rewards: 0.0 Last height farmed: 0 Local Harvester 1310 plots of size: 129.671 TiB Plot count for all harvesters: 1310 Total size of plots: 129.671 TiB Estimated network space: 35.727 EiB Expected time to win: 2 months Note: log into your key using ‘chia wallet show’ to see rewards for each key Mon 30 Aug 2021 04:41:01 AM MST —————————————————————- – Chia Show Network Connections —————————————————————- FULL_NODE Connextions 58 WALLET Connextions 2 Mon 30 Aug 2021 04:41:01 AM MST —————————————————————- – Drive Temp —————————————————————- 08.30.21.04.40:/dev/sda:33 08.30.21.04.40:/dev/sdb:34 08.30.21.04.40:/dev/sdc:33 08.30.21.04.40:/dev/sdd:34 08.30.21.04.40:/dev/sde:34 08.30.21.04.40:/dev/sdf:31 08.30.21.04.40:/dev/sdg:39 08.30.21.04.40:/dev/sdh:41 08.30.21.04.40:/dev/sdi:41 08.30.21.04.40:/dev/sdj:40 08.30.21.04.40:/dev/sdk:43 Mon 30 Aug 2021 04:41:01 AM MST —————————————————————- – Plot Details —————————————————————- 11T /nas/n1a/plots 9.9T /nas/n1b/plots 1.8T /nas/n2a/plots 1.9T /nas/n2b/plots 12T /nas/n4a/plots 12T /nas/n4b/plots 1.3T /nas/sc1a/plots 3.6T /nas/sc1b/plots 3.6T /nas/sc1c/plots 3.6T /nas/sc1d/plots 3.6T /nas/sc1e/plots 609G /nas/zero/plots 811G /nas/zero2/plots 7.2T /nas/a/plots 7.0T /nas/b/plots 406G /nas/c/plots 3.3T /nas/d/plots 3.3T /nas/e/plots 913G /nas/f/plots 11T /nas/g/plots 11T /nas/h/plots 11T /nas/i/plots 3.6T /nas/j/plots 3.6T /nas/k/plots Mon 30 Aug 2021 04:41:01 AM MST —————————————————————- – Drive Free —————————————————————- //cp1/zero 1913454044 1754140992 159313052 92% /home/zero //cp1/zero2 1441098196 1316590244 124507952 92% /home/zero2 /dev/sda1 7751366404 7652595324 98754696 99% /nas/a /dev/sdb1 7751366404 7613526628 137823392 99% /nas/b /dev/sdd1 3844659208 3677526672 167116152 96% /nas/d /dev/sde1 3844661068 3683797692 160846992 96% /nas/e /dev/sdf1 960385056 956600380 3768292 100% /nas/f /dev/sdg1 11625432492 11585587080 39829028 100% /nas/g /dev/sdh1 11625432492 11597845396 27570712 100% /nas/h /dev/sdi1 11625432492 11603558940 21857168 100% /nas/i /dev/sdj1 3844629500 3826289616 18323500 100% /nas/j /dev/sdk1 3844640564 3825653056 18971124 100% /nas/k //n1/n1a 31130140152 24618514728 6511625424 80% /nas/n1a //n1/n1b 31130140152 24304991880 6825148272 79% /nas/n1b //n2/n2a 7752457832 7262403900 490053932 94% /nas/n2a //n2/n2b 7752326760 7387896148 364430612 96% /nas/n2b //n3/n3a 3844660232 3826219484 18440748 100% /nas/n3a //n3/n3b 960380628 744113508 216267120 78% /nas/n3b //n4/n4a 58359505136 51391752944 6967752192 89% /nas/n4a //n4/n4b 58359505136 50772609888 7586895248 87% /nas/n4b //sc1/sc1a 3834087900 2905899404 928188496 76% /nas/sc1a //sc1/sc1b 3845578572 3826587384 18991188 100% /nas/sc1b //sc1/sc1c 3845578572 3826368252 19210320 100% /nas/sc1c //sc1/sc1d 3845578572 3826454204 19124368 100% /nas/sc1d //sc1/sc1e 3845578572 3833644088 11934484 100% /nas/sc1e /dev/sdc2 1921285344 632693660 1190925892 35% / Mon 30 Aug 2021 04:41:01 AM MST —————————————————————- – Memory Information —————————————————————- MemTotal: 32764796 kB MemFree: 302560 kB MemAvailable: 25767792 kB Buffers: 534716 kB Cached: 23971440 kB SwapCached: 483960 kB Active: 6342728 kB Inactive: 24266284 kB Active(anon): 4814680 kB Inactive(anon): 1297852 kB Mon 30 Aug 2021 04:41:01 AM MST —————————————————————- – DMESG | Tail - 10 —————————————————————- [858876.773836] CIFS VFS: n4 has not responded in 180 seconds. Reconnecting… [858877.285345] CIFS VFS: n2 has not responded in 180 seconds. Reconnecting… [858877.285366] CIFS VFS: n2 has not responded in 180 seconds. Reconnecting… [858877.797376] CIFS VFS: n1 has not responded in 180 seconds. Reconnecting… [858877.797400] CIFS VFS: cp1 has not responded in 180 seconds. Reconnecting… [858879.333476] CIFS VFS: n3 has not responded in 180 seconds. Reconnecting… [858879.333479] CIFS VFS: n3 has not responded in 180 seconds. Reconnecting… [931444.226591] EXT4-fs (sda1): error count since last fsck: 70 [931444.226605] EXT4-fs (sda1): initial error at time 1628798920: ext4_journal_check_start:61 [931444.226608] EXT4-fs (sda1): last error at time 1629311738: ext4_discard_preallocations:4094 Mon 30 Aug 2021 04:41:01 AM MST —————————————————————- – PSCPU10 | HEAD - 10 —————————————————————- root 1594572 61.9 4.1 1473596 1354392 tty1 DL Aug28 1211:11 /root/chia-blockchain/venv/bin/python /root/chia-blockchain/venv/bin/chia plots create -n 15 -r 2 -t /nas/b/chia/ -2 /nas/b/chia/ -d /nas/n1a/plots/ root 1594574 61.8 4.1 1473504 1354948 tty1 RL Aug28 1210:38 /root/chia-blockchain/venv/bin/python /root/chia-blockchain/venv/bin/chia plots create -n 10 -r 2 -t /nas/e/chia/ -2 /nas/e/chia/ -d /nas/c/plots/ root 1594573 51.7 0.1 252488 42936 tty1 DL Aug28 1012:22 /root/chia-blockchain/venv/bin/python /root/chia-blockchain/venv/bin/chia plots create -n 15 -r 2 -t /nas/d/chia/ -2 /nas/d/chia/ -d /nas/n1b/plots/ root 3075 6.5 5.5 2574676 1814172 ? Sl Aug18 1108:28 _ chia_full_node root 3073 1.2 0.1 3798812 59676 ? Sl Aug18 212:19 _ chia_harvester root 3074 1.0 0.1 306628 52540 ? SLl Aug18 175:19 _ chia_farmer root 175 1.0 0.0 0 0 ? S Aug18 176:35 _ [kswapd0] root 1831775 0.7 0.0 0 0 ? I 04:15 0:11 _ [kworker/u32:3-events_power_efficient] root 1828134 0.7 0.0 0 0 ? S 04:01 0:18 _ [kworker/u32:5+flush-cifs-3] root 1821438 0.5 0.0 0 0 ? I 02:42 0:40 _ [kworker/u32:0-events_unbound] Mon 30 Aug 2021 04:41:01 AM MST —————————————————————- – Chia show PLOTing Processeses —————————————————————- 4 R root 1594572 1 61 80 0 - 368399 - Aug28 tty1 20:11:11 /root/chia-blockchain/venv/bin/python /root/chia-blockchain/venv/bin/chia plots create -n 15 -r 2 -t /nas/b/chia/ -2 /nas/b/chia/ -d /nas/n1a/plots/ 4 D root 1594573 1 51 80 0 - 63122 generi Aug28 tty1 16:52:22 /root/chia-blockchain/venv/bin/python /root/chia-blockchain/venv/bin/chia plots create -n 15 -r 2 -t /nas/d/chia/ -2 /nas/d/chia/ -d /nas/n1b/plots/ 4 D root 1594574 1 61 80 0 - 368376 generi Aug28 tty1 20:10:38 /root/chia-blockchain/venv/bin/python /root/chia-blockchain/venv/bin/chia plots create -n 10 -r 2 -t /nas/e/chia/ -2 /nas/e/chia/ -d /nas/c/plots/ Mon 30 Aug 2021 04:41:01 AM MST —————————————————————- – Log Eligible & How Long —————————————————————- total 213816 -rw-r–r– 1 root root 9034328 Aug 30 04:41 debug.log -rw-r–r– 1 root root 20971610 Aug 29 23:54 debug.log.1 -rw-r–r– 1 root root 20971619 Aug 23 08:27 debug.log.10 -rw-r–r– 1 root root 20971615 Aug 29 10:00 debug.log.2 -rw-r–r– 1 root root 20971624 Aug 28 23:06 debug.log.3 -rw-r–r– 1 root root 20971540 Aug 28 02:26 debug.log.4 -rw-r–r– 1 root root 20971523 Aug 27 05:19 debug.log.5 -rw-r–r– 1 root root 20971822 Aug 26 09:53 debug.log.6 -rw-r–r– 1 root root 20971665 Aug 25 16:06 debug.log.7 -rw-r–r– 1 root root 20971532 Aug 24 22:00 debug.log.8 -rw-r–r– 1 root root 20971529 Aug 24 04:15 debug.log.9 ——————————————————————- debug 0: 1760 debug 1: 4908 debug 2: 3649 debug 3: 7887 debug 4: 8125 debug 5: 7439 debug 6: 6796 debug 7: 6876 debug 8: 6764 debug 9: 7582 debug 10: 6876 Mon 30 Aug 2021 04:41:01 AM MST —————————————————————- – PLOT TIMES —————————————————————- Drive a Drive b Total time = 8.95597 Hours CPU (61.960%) Sun Aug 29 05:02:54 2021 Total time = 9.02444 Hours CPU (61.140%) Sun Aug 29 14:21:16 2021 Total time = 8.98715 Hours CPU (61.490%) Sun Aug 29 23:37:27 2021 Drive c Drive d Total time = 10.5758 Hours CPU (53.410%) Sun Aug 29 06:40:05 2021 Total time = 10.53 Hours CPU (53.200%) Sun Aug 29 17:29:42 2021 Drive e Total time = 8.94114 Hours CPU (61.760%) Sun Aug 29 05:02:01 2021 Total time = 8.75893 Hours CPU (62.330%) Sun Aug 29 14:06:19 2021 Total time = 8.95444 Hours CPU (62.070%) Sun Aug 29 23:23:50 2021 Drive f Mon 30 Aug 2021 04:41:01 AM MST —————————————————————- – PLOTs PER DAY —————————————————————- 04.29.2021 = 2 04.30.2021 = 3 05.01.2021 = 3 05.02.2021 = 5 05.03.2021 = 4 05.06.2021 = 4 05.07.2021 = 6 05.08.2021 = 8 05.09.2021 = 8 05.10.2021 = 5 05.11.2021 = 6 05.12.2021 = 5 05.13.2021 = 4 05.14.2021 = 4 05.15.2021 = 4 05.16.2021 = 6 05.17.2021 = 1 05.18.2021 = 4 05.19.2021 = 8 05.20.2021 = 9 05.21.2021 = 2 05.22.2021 = 2 05.23.2021 = 4 05.24.2021 = 5 05.25.2021 = 8 05.26.2021 = 7 05.27.2021 = 9 05.28.2021 = 9 05.29.2021 = 8 05.30.2021 = 7 05.31.2021 = 7 06.01.2021 = 8 06.02.2021 = 4 06.03.2021 = 6 06.04.2021 = 9 06.05.2021 = 12 06.06.2021 = 10 06.07.2021 = 10 06.08.2021 = 9 06.09.2021 = 14 06.10.2021 = 9 06.11.2021 = 9 06.12.2021 = 8 06.13.2021 = 9 06.14.2021 = 10 06.15.2021 = 11 06.16.2021 = 11 06.17.2021 = 13 06.18.2021 = 13 06.19.2021 = 12 06.20.2021 = 17 06.21.2021 = 15 06.22.2021 = 12 06.23.2021 = 10 06.24.2021 = 12 06.25.2021 = 16 06.26.2021 = 9 06.27.2021 = 12 06.28.2021 = 15 06.29.2021 = 17 06.30.2021 = 12 07.01.2021 = 15 07.02.2021 = 16 07.03.2021 = 15 07.04.2021 = 21 07.05.2021 = 19 07.06.2021 = 17 07.07.2021 = 15 07.08.2021 = 8 07.09.2021 = 4 07.10.2021 = 8 07.11.2021 = 5 07.12.2021 = 8 07.13.2021 = 8 07.14.2021 = 6 07.15.2021 = 4 07.16.2021 = 7 07.17.2021 = 10 07.18.2021 = 6 07.19.2021 = 3 07.20.2021 = 7 07.21.2021 = 8 07.22.2021 = 10 07.23.2021 = 7 07.24.2021 = 7 07.25.2021 = 10 07.26.2021 = 14 07.27.2021 = 4 07.28.2021 = 7 07.29.2021 = 8 07.30.2021 = 7 07.31.2021 = 8 08.01.2021 = 9 08.02.2021 = 7 08.03.2021 = 7 08.04.2021 = 5 08.05.2021 = 5 08.06.2021 = 5 08.07.2021 = 4 08.08.2021 = 3 08.10.2021 = 3 08.11.2021 = 2 08.12.2021 = 3 08.13.2021 = 2 08.14.2021 = 2 08.15.2021 = 2 08.18.2021 = 2 08.19.2021 = 5 08.20.2021 = 4 08.21.2021 = 6 08.22.2021 = 4 08.23.2021 = 1 08.24.2021 = 3 08.28.2021 = 3 08.29.2021 = 5 Mon 30 Aug 2021 04:41:01 AM MST —————————————————————- – Plot Total Times [a-h] —————————————————————- /var/log/plotb.log:Total time = 32241.492 seconds. CPU (61.960%) Sun Aug 29 05:02:54 2021 /var/log/plotb.log:Total time = 32488.000 seconds. CPU (61.140%) Sun Aug 29 14:21:16 2021 /var/log/plotb.log:Total time = 32353.748 seconds. CPU (61.490%) Sun Aug 29 23:37:27 2021 /var/log/plotd.log:Total time = 38072.746 seconds. CPU (53.410%) Sun Aug 29 06:40:05 2021 /var/log/plotd.log:Total time = 37908.051 seconds. CPU (53.200%) Sun Aug 29 17:29:42 2021 /var/log/plote.log:Total time = 32188.105 seconds. CPU (61.760%) Sun Aug 29 05:02:01 2021 /var/log/plote.log:Total time = 31532.149 seconds. CPU (62.330%) Sun Aug 29 14:06:19 2021 /var/log/plote.log:Total time = 32235.976 seconds. CPU (62.070%) Sun Aug 29 23:23:50 2021 Mon 30 Aug 2021 04:41:01 AM MST —————————————————————- – Plot Process per drive —————————————————————- Mon 30 Aug 2021 04:41:01 AM MST Drive a: Last:10 —————————————————————— Mon 30 Aug 2021 04:41:01 AM MST Drive b: Last:10 —————————————————————— Time for phase 1 = 13360.807 seconds. CPU (80.830%) Sun Aug 29 18:20:54 2021 Starting phase 2/4: Backpropagation into tmp files… Sun Aug 29 18:20:54 2021 Time for phase 2 = 5862.213 seconds. CPU (52.650%) Sun Aug 29 19:58:36 2021 Starting phase 3/4: Compression from tmp files into “/nas/b/chia/plot-k32-2021-08-29-14-38-a10cbba19fd7b08d3d161cc7471b9c86cea9d2c907ca3681f6b81d3c45345274.plot.2.tmp” … Sun Aug 29 19:58:36 2021 Time for phase 3 = 12513.239 seconds. CPU (45.010%) Sun Aug 29 23:27:09 2021 Starting phase 4/4: Write Checkpoint tables into “/nas/b/chia/plot-k32-2021-08-29-14-38-a10cbba19fd7b08d3d161cc7471b9c86cea9d2c907ca3681f6b81d3c45345274.plot.2.tmp” … Sun Aug 29 23:27:09 2021 Time for phase 4 = 617.487 seconds. CPU (60.720%) Sun Aug 29 23:37:27 2021 Starting phase 1/4: Forward Propagation into tmp files… Sun Aug 29 23:54:21 2021 Time for phase 1 = 13477.976 seconds. CPU (80.980%) Mon Aug 30 03:38:59 2021 Starting phase 2/4: Backpropagation into tmp files… Mon Aug 30 03:38:59 2021 Mon 30 Aug 2021 04:41:01 AM MST Drive c: Last:10 —————————————————————— Mon 30 Aug 2021 04:41:01 AM MST Drive d: Last:10 —————————————————————— Starting phase 4/4: Write Checkpoint tables into “/nas/d/chia/plot-k32-2021-08-29-06-57-31693f42d45c99497ebd5fec38a248225627d1f48481ec83762bacccfd1bb83e.plot.2.tmp” … Sun Aug 29 17:16:03 2021 Time for phase 4 = 818.452 seconds. CPU (48.070%) Sun Aug 29 17:29:42 2021 Starting phase 1/4: Forward Propagation into tmp files… Sun Aug 29 17:47:46 2021 Time for phase 1 = 16527.715 seconds. CPU (64.920%) Sun Aug 29 22:23:14 2021 Starting phase 2/4: Backpropagation into tmp files… Sun Aug 29 22:23:14 2021 Time for phase 2 = 6313.381 seconds. CPU (49.460%) Mon Aug 30 00:08:27 2021 Starting phase 3/4: Compression from tmp files into “/nas/d/chia/plot-k32-2021-08-29-17-47-8068074c61b57228ab614da814083ab401412b26a9aab839794a90659b7f651d.plot.2.tmp” … Mon Aug 30 00:08:27 2021 Time for phase 3 = 14992.380 seconds. CPU (38.580%) Mon Aug 30 04:18:19 2021 Starting phase 4/4: Write Checkpoint tables into “/nas/d/chia/plot-k32-2021-08-29-17-47-8068074c61b57228ab614da814083ab401412b26a9aab839794a90659b7f651d.plot.2.tmp” … Mon Aug 30 04:18:19 2021 Time for phase 4 = 743.191 seconds. CPU (51.000%) Mon Aug 30 04:30:43 2021 Mon 30 Aug 2021 04:41:01 AM MST Drive e: Last:10 —————————————————————— Time for phase 1 = 13671.927 seconds. CPU (79.870%) Sun Aug 29 18:14:26 2021 Starting phase 2/4: Backpropagation into tmp files… Sun Aug 29 18:14:26 2021 Time for phase 2 = 5700.398 seconds. CPU (54.520%) Sun Aug 29 19:49:26 2021 Starting phase 3/4: Compression from tmp files into “/nas/e/chia/plot-k32-2021-08-29-14-26-a8c828506ec96c8e4370416d88b2604918cc99080d1ce1ae9ae56c8a88565365.plot.2.tmp” … Sun Aug 29 19:49:26 2021 Time for phase 3 = 12184.688 seconds. CPU (45.920%) Sun Aug 29 23:12:31 2021 Starting phase 4/4: Write Checkpoint tables into “/nas/e/chia/plot-k32-2021-08-29-14-26-a8c828506ec96c8e4370416d88b2604918cc99080d1ce1ae9ae56c8a88565365.plot.2.tmp” … Sun Aug 29 23:12:31 2021 Time for phase 4 = 678.961 seconds. CPU (57.110%) Sun Aug 29 23:23:50 2021 Starting phase 1/4: Forward Propagation into tmp files… Sun Aug 29 23:46:56 2021 Time for phase 1 = 13790.544 seconds. CPU (78.390%) Mon Aug 30 03:36:47 2021 Starting phase 2/4: Backpropagation into tmp files… Mon Aug 30 03:36:47 2021 Mon 30 Aug 2021 04:41:01 AM MST Drive f: Last:10 —————————————————————— Mon 30 Aug 2021 04:41:01 AM MST —————————————————————- – CHIA Debug Log Warning / Error (LAST 10) —————————————————————- -rw-r–r– 1 root root 9036178 Aug 30 04:41 /root/.chia/mainnet/log/debug.log —————————————————————— FOUND Proofs : Mon 30 Aug 2021 04:41:01 AM MST —————————————————————- – CHIA Debug Log ( 1 ) Warning / Error LAST 10 —————————————————————- -rw-r–r– 1 root root 20971610 Aug 29 23:54 /root/.chia/mainnet/log/debug.log.1 —————————————————————— FOUND Proofs : 2021-08-29T12:17:04.388 full_node full_node_server : ERROR Exception: [Errno 110] Connection timed out with 182.89.89.40 2021-08-29T14:37:59.723 farmer chia.farmer.farmer : ERROR Harvester did not respond. You might need to update harvester to the latest version 2021-08-29T17:06:37.572 full_node full_node_server : ERROR Exception: [Errno 110] Connection timed out with 173.28.202.79 2021-08-29T23:38:22.077 full_node chia.full_node.full_node: ERROR got weight proof request for unknown peak c0d69841dd7a1946a55057f37dc88d8500a233feeb9ae35f4dbe6d9c132e5df3 Mon 30 Aug 2021 04:41:01 AM MST —————————————————————- – CHIA Debug Log ( 2 ) Warning / Error LAST 10 —————————————————————- -rw-r–r– 1 root root 20971615 Aug 29 10:00 /root/.chia/mainnet/log/debug.log.2 —————————————————————— FOUND Proofs : 2021-08-29T00:12:02.820 full_node full_node_server : ERROR Exception: [Errno 110] Connection timed out with 111.40.59.242 2021-08-29T05:18:14.070 full_node chia.full_node.full_node: ERROR got weight proof request for unknown peak c51cf7f8964492ef67ae0341c3bd02709aacf14226241b261e9c8f45ffccaad6 2021-08-29T06:11:18.020 full_node full_node_server : ERROR Exception: [Errno 110] Connection timed out with 115.93.222.234 2021-08-29T09:14:50.116 full_node full_node_server : ERROR Exception: [Errno 110] Connection timed out with 114.225.91.68 Mon 30 Aug 2021 04:41:01 AM MST —————————————————————- – CHIA Debug Log ( 3 ) Warning / Error LAST 10 —————————————————————- -rw-r–r– 1 root root 20971624 Aug 28 23:06 /root/.chia/mainnet/log/debug.log.3 —————————————————————— FOUND Proofs : Mon 30 Aug 2021 04:41:01 AM MST —————————————————————- – CHIA Debug Log ( 4 ) Warning / Error LAST 10 —————————————————————- -rw-r–r– 1 root root 20971540 Aug 28 02:26 /root/.chia/mainnet/log/debug.log.4 —————————————————————— FOUND Proofs : 2021-08-27T07:43:33.765 full_node full_node_server : ERROR Exception: [Errno 110] Connection timed out with 110.87.81.123 2021-08-27T08:15:53.226 full_node full_node_server : ERROR Exception: [Errno 110] Connection timed out with 120.230.117.188 2021-08-27T09:22:12.484 full_node full_node_server : ERROR Exception: [Errno 110] Connection timed out with 176.33.69.46 2021-08-27T10:04:25.861 full_node full_node_server : ERROR Exception: [Errno 110] Connection timed out with 101.84.52.157 2021-08-27T11:29:09.904 harvester chia.harvester.harvester: ERROR Error using prover object badbit or failbit after reading size 8 at position 85292863315 2021-08-27T11:29:09.935 harvester chia.harvester.harvester: ERROR File: /nas/a/plots/plot-k32-2021-06-29-22-27-f488f56c75a93231e3942e3186880e4edfcc8bd4ef0ceda6f110800ad6f1220b.plot Plot ID: f488f56c75a93231e3942e3186880e4edfcc8bd4ef0ceda6f110800ad6f1220b, challenge: 66fe8b54531fd93788a1e53a61bf1000cd7c4ac8c8e5c73f596c1b4288e96693, plot_info: PlotInfo(prover=, pool_public_key=, pool_contract_puzzle_hash=None, plot_public_key=, file_size=108791552003, time_modified=1625034563.0319898) Mon 30 Aug 2021 04:41:01 AM MST —————————————————————- – CHIA Debug Log ( 5 ) Warning / Error LAST 10 —————————————————————- -rw-r–r– 1 root root 20971523 Aug 27 05:19 /root/.chia/mainnet/log/debug.log.5 —————————————————————— FOUND Proofs : 2021-08-26T10:30:00.837 full_node full_node_server : ERROR Exception: [Errno 110] Connection timed out with 182.126.202.236 Mon 30 Aug 2021 04:41:01 AM MST —————————————————————- – CHIA Debug Log ( 6 ) Warning / Error LAST 10 —————————————————————- -rw-r–r– 1 root root 20971822 Aug 26 09:53 /root/.chia/mainnet/log/debug.log.6 —————————————————————— FOUND Proofs : 2021-08-26T02:08:45.956 full_node full_node_server : ERROR Exception: [Errno 110] Connection timed out with 31.60.15.75 2021-08-26T03:42:11.332 full_node full_node_server : ERROR Exception: [Errno 110] Connection timed out with 5.173.241.35 Mon 30 Aug 2021 04:41:01 AM MST —————————————————————- – CHIA Debug Log ( 7 ) Warning / Error LAST 10 —————————————————————- -rw-r–r– 1 root root 20971665 Aug 25 16:06 /root/.chia/mainnet/log/debug.log.7 —————————————————————— FOUND Proofs : 2021-08-25T05:21:42.277 full_node full_node_server : ERROR Exception: [Errno 110] Connection timed out with 123.155.68.6 Mon 30 Aug 2021 04:41:01 AM MST —————————————————————- – CHIA Debug Log ( 8 ) Warning / Error LAST 10 —————————————————————- -rw-r–r– 1 root root 20971532 Aug 24 22:00 /root/.chia/mainnet/log/debug.log.8 —————————————————————— FOUND Proofs : Mon 30 Aug 2021 04:41:01 AM MST —————————————————————- – CHIA Debug Log ( 9 ) Warning / Error LAST 10 —————————————————————- -rw-r–r– 1 root root 20971529 Aug 24 04:15 /root/.chia/mainnet/log/debug.log.9 —————————————————————— FOUND Proofs : Mon 30 Aug 2021 04:41:01 AM MST —————————————————————- – CHIA Debug Log ( 10 ) Warning / Error LAST 10 —————————————————————- -rw-r–r– 1 root root 20971619 Aug 23 08:27 /root/.chia/mainnet/log/debug.log.10 —————————————————————— FOUND Proofs : Mon 30 Aug 2021 04:41:01 AM MST —————————————————————- – Total LOG Eligible Last 10 log files = 68662 —————————————————————- Mon 30 Aug 2021 04:41:01 AM MST —————————————————————- – Chia Debug Plots taking too long all log files —————————————————————- LONGPLOTS = 2278 TOTAL Plots = 68662 Mon 30 Aug 2021 04:41:01 AM MST —————————————————————- – Count by time taken —————————————————————- 5 Sec - 750 CNT - 33 % Longplots 1.1 % Total 6 Sec - 523 CNT - 23 % Longplots 0.76 % Total 7 Sec - 135 CNT - 5.9 % Longplots 0.2 % Total 8 Sec - 93 CNT - 4.1 % Longplots 0.14 % Total 9 Sec - 42 CNT - 1.8 % Longplots 0.061 % Total 10 Sec - 46 CNT - 2 % Longplots 0.067 % Total 11 Sec - 138 CNT - 6.1 % Longplots 0.2 % Total 12 Sec - 50 CNT - 2.2 % Longplots 0.073 % Total 13 Sec - 38 CNT - 1.7 % Longplots 0.055 % Total 14 Sec - 19 CNT - 0.83 % Longplots 0.028 % Total 15 Sec - 27 CNT - 1.2 % Longplots 0.039 % Total 16 Sec - 44 CNT - 1.9 % Longplots 0.064 % Total 17 Sec - 46 CNT - 2 % Longplots 0.067 % Total 18 Sec - 27 CNT - 1.2 % Longplots 0.039 % Total 19 Sec - 21 CNT - 0.92 % Longplots 0.031 % Total 20 Sec - 5 CNT - 0.22 % Longplots 0.0073 % Total 21 Sec - 4 CNT - 0.18 % Longplots 0.0058 % Total 22 Sec - 40 CNT - 1.8 % Longplots 0.058 % Total 23 Sec - 26 CNT - 1.1 % Longplots 0.038 % Total 24 Sec - 3 CNT - 0.13 % Longplots 0.0044 % Total 25 Sec - 13 CNT - 0.57 % Longplots 0.019 % Total 27 Sec - 24 CNT - 1.1 % Longplots 0.035 % Total 28 Sec - 10 CNT - 0.44 % Longplots 0.015 % Total 29 Sec - 21 CNT - 0.92 % Longplots 0.031 % Total 30 Sec - 3 CNT - 0.13 % Longplots 0.0044 % Total 31 Sec - 2 CNT - 0.088 % Longplots 0.0029 % Total 33 Sec - 24 CNT - 1.1 % Longplots 0.035 % Total 34 Sec - 5 CNT - 0.22 % Longplots 0.0073 % Total 35 Sec - 2 CNT - 0.088 % Longplots 0.0029 % Total 36 Sec - 5 CNT - 0.22 % Longplots 0.0073 % Total 38 Sec - 9 CNT - 0.4 % Longplots 0.013 % Total 39 Sec - 5 CNT - 0.22 % Longplots 0.0073 % Total 40 Sec - 9 CNT - 0.4 % Longplots 0.013 % Total 41 Sec - 8 CNT - 0.35 % Longplots 0.012 % Total 42 Sec - 3 CNT - 0.13 % Longplots 0.0044 % Total 44 Sec - 1 CNT - 0.044 % Longplots 0.0015 % Total 46 Sec - 3 CNT - 0.13 % Longplots 0.0044 % Total 47 Sec - 1 CNT - 0.044 % Longplots 0.0015 % Total 48 Sec - 2 CNT - 0.088 % Longplots 0.0029 % Total 49 Sec - 2 CNT - 0.088 % Longplots 0.0029 % Total 51 Sec - 6 CNT - 0.26 % Longplots 0.0087 % Total 52 Sec - 3 CNT - 0.13 % Longplots 0.0044 % Total 57 Sec - 6 CNT - 0.26 % Longplots 0.0087 % Total 63 Sec - 2 CNT - 0.088 % Longplots 0.0029 % Total 69 Sec - 1 CNT - 0.044 % Longplots 0.0015 % Total 73 Sec - 2 CNT - 0.088 % Longplots 0.0029 % Total 79 Sec - 1 CNT - 0.044 % Longplots 0.0015 % Total 85 Sec - 3 CNT - 0.13 % Longplots 0.0044 % Total 89 Sec - 1 CNT - 0.044 % Longplots 0.0015 % Total 91 Sec - 3 CNT - 0.13 % Longplots 0.0044 % Total 92 Sec - 2 CNT - 0.088 % Longplots 0.0029 % Total 97 Sec - 5 CNT - 0.22 % Longplots 0.0073 % Total 132 Sec - 2 CNT - 0.088 % Longplots 0.0029 % Total 153 Sec - 1 CNT - 0.044 % Longplots 0.0015 % Total 166 Sec - 1 CNT - 0.044 % Longplots 0.0015 % Total 189 Sec - 1 CNT - 0.044 % Longplots 0.0015 % Total 206 Sec - 1 CNT - 0.044 % Longplots 0.0015 % Total 213 Sec - 1 CNT - 0.044 % Longplots 0.0015 % Total 297 Sec - 1 CNT - 0.044 % Longplots 0.0015 % Total 298 Sec - 1 CNT - 0.044 % Longplots 0.0015 % Total 430 Sec - 1 CNT - 0.044 % Longplots 0.0015 % Total 503 Sec - 1 CNT - 0.044 % Longplots 0.0015 % Total 524 Sec - 1 CNT - 0.044 % Longplots 0.0015 % Total 526 Sec - 1 CNT - 0.044 % Longplots 0.0015 % Total 528 Sec - 1 CNT - 0.044 % Longplots 0.0015 % Total