1*6236dae4SAndroid Build Coastguard Worker#!/usr/bin/env perl 2*6236dae4SAndroid Build Coastguard Worker#*************************************************************************** 3*6236dae4SAndroid Build Coastguard Worker# _ _ ____ _ 4*6236dae4SAndroid Build Coastguard Worker# Project ___| | | | _ \| | 5*6236dae4SAndroid Build Coastguard Worker# / __| | | | |_) | | 6*6236dae4SAndroid Build Coastguard Worker# | (__| |_| | _ <| |___ 7*6236dae4SAndroid Build Coastguard Worker# \___|\___/|_| \_\_____| 8*6236dae4SAndroid Build Coastguard Worker# 9*6236dae4SAndroid Build Coastguard Worker# Copyright (C) Daniel Stenberg, <[email protected]>, et al. 10*6236dae4SAndroid Build Coastguard Worker# 11*6236dae4SAndroid Build Coastguard Worker# This software is licensed as described in the file COPYING, which 12*6236dae4SAndroid Build Coastguard Worker# you should have received as part of this distribution. The terms 13*6236dae4SAndroid Build Coastguard Worker# are also available at https://curl.se/docs/copyright.html. 14*6236dae4SAndroid Build Coastguard Worker# 15*6236dae4SAndroid Build Coastguard Worker# You may opt to use, copy, modify, merge, publish, distribute and/or sell 16*6236dae4SAndroid Build Coastguard Worker# copies of the Software, and permit persons to whom the Software is 17*6236dae4SAndroid Build Coastguard Worker# furnished to do so, under the terms of the COPYING file. 18*6236dae4SAndroid Build Coastguard Worker# 19*6236dae4SAndroid Build Coastguard Worker# This software is distributed on an "AS IS" basis, WITHOUT WARRANTY OF ANY 20*6236dae4SAndroid Build Coastguard Worker# KIND, either express or implied. 21*6236dae4SAndroid Build Coastguard Worker# 22*6236dae4SAndroid Build Coastguard Worker# SPDX-License-Identifier: curl 23*6236dae4SAndroid Build Coastguard Worker# 24*6236dae4SAndroid Build Coastguard Worker########################################################################### 25*6236dae4SAndroid Build Coastguard Worker 26*6236dae4SAndroid Build Coastguard Worker# For documentation, run `man ./runtests.1` and see README.md. 27*6236dae4SAndroid Build Coastguard Worker 28*6236dae4SAndroid Build Coastguard Worker# Experimental hooks are available to run tests remotely on machines that 29*6236dae4SAndroid Build Coastguard Worker# are able to run curl but are unable to run the test harness. 30*6236dae4SAndroid Build Coastguard Worker# The following sections need to be modified: 31*6236dae4SAndroid Build Coastguard Worker# 32*6236dae4SAndroid Build Coastguard Worker# $HOSTIP, $HOST6IP - Set to the address of the host running the test suite 33*6236dae4SAndroid Build Coastguard Worker# $CLIENTIP, $CLIENT6IP - Set to the address of the host running curl 34*6236dae4SAndroid Build Coastguard Worker# runclient, runclientoutput - Modify to copy all the files in the log/ 35*6236dae4SAndroid Build Coastguard Worker# directory to the system running curl, run the given command remotely 36*6236dae4SAndroid Build Coastguard Worker# and save the return code or returned stdout (respectively), then 37*6236dae4SAndroid Build Coastguard Worker# copy all the files from the remote system's log/ directory back to 38*6236dae4SAndroid Build Coastguard Worker# the host running the test suite. This can be done a few ways, such 39*6236dae4SAndroid Build Coastguard Worker# as using scp & ssh, rsync & telnet, or using a NFS shared directory 40*6236dae4SAndroid Build Coastguard Worker# and ssh. 41*6236dae4SAndroid Build Coastguard Worker# 42*6236dae4SAndroid Build Coastguard Worker# 'make && make test' needs to be done on both machines before making the 43*6236dae4SAndroid Build Coastguard Worker# above changes and running runtests.pl manually. In the shared NFS case, 44*6236dae4SAndroid Build Coastguard Worker# the contents of the tests/server/ directory must be from the host 45*6236dae4SAndroid Build Coastguard Worker# running the test suite, while the rest must be from the host running curl. 46*6236dae4SAndroid Build Coastguard Worker# 47*6236dae4SAndroid Build Coastguard Worker# Note that even with these changes a number of tests will still fail (mainly 48*6236dae4SAndroid Build Coastguard Worker# to do with cookies, those that set environment variables, or those that 49*6236dae4SAndroid Build Coastguard Worker# do more than touch the file system in a <precheck> or <postcheck> 50*6236dae4SAndroid Build Coastguard Worker# section). These can be added to the $TESTCASES line below, 51*6236dae4SAndroid Build Coastguard Worker# e.g. $TESTCASES="!8 !31 !63 !cookies..." 52*6236dae4SAndroid Build Coastguard Worker# 53*6236dae4SAndroid Build Coastguard Worker# Finally, to properly support -g and -n, checktestcmd needs to change 54*6236dae4SAndroid Build Coastguard Worker# to check the remote system's PATH, and the places in the code where 55*6236dae4SAndroid Build Coastguard Worker# the curl binary is read directly to determine its type also need to be 56*6236dae4SAndroid Build Coastguard Worker# fixed. As long as the -g option is never given, and the -n is always 57*6236dae4SAndroid Build Coastguard Worker# given, this won't be a problem. 58*6236dae4SAndroid Build Coastguard Worker 59*6236dae4SAndroid Build Coastguard Workeruse strict; 60*6236dae4SAndroid Build Coastguard Worker# Promote all warnings to fatal 61*6236dae4SAndroid Build Coastguard Workeruse warnings FATAL => 'all'; 62*6236dae4SAndroid Build Coastguard Workeruse 5.006; 63*6236dae4SAndroid Build Coastguard Workeruse POSIX qw(strftime); 64*6236dae4SAndroid Build Coastguard Worker 65*6236dae4SAndroid Build Coastguard Worker# These should be the only variables that might be needed to get edited: 66*6236dae4SAndroid Build Coastguard Worker 67*6236dae4SAndroid Build Coastguard WorkerBEGIN { 68*6236dae4SAndroid Build Coastguard Worker # Define srcdir to the location of the tests source directory. This is 69*6236dae4SAndroid Build Coastguard Worker # usually set by the Makefile, but for out-of-tree builds with direct 70*6236dae4SAndroid Build Coastguard Worker # invocation of runtests.pl, it may not be set. 71*6236dae4SAndroid Build Coastguard Worker if(!defined $ENV{'srcdir'}) { 72*6236dae4SAndroid Build Coastguard Worker use File::Basename; 73*6236dae4SAndroid Build Coastguard Worker $ENV{'srcdir'} = dirname(__FILE__); 74*6236dae4SAndroid Build Coastguard Worker } 75*6236dae4SAndroid Build Coastguard Worker push(@INC, $ENV{'srcdir'}); 76*6236dae4SAndroid Build Coastguard Worker # run time statistics needs Time::HiRes 77*6236dae4SAndroid Build Coastguard Worker eval { 78*6236dae4SAndroid Build Coastguard Worker no warnings "all"; 79*6236dae4SAndroid Build Coastguard Worker require Time::HiRes; 80*6236dae4SAndroid Build Coastguard Worker import Time::HiRes qw( time ); 81*6236dae4SAndroid Build Coastguard Worker } 82*6236dae4SAndroid Build Coastguard Worker} 83*6236dae4SAndroid Build Coastguard Worker 84*6236dae4SAndroid Build Coastguard Workeruse Digest::MD5 qw(md5); 85*6236dae4SAndroid Build Coastguard Workeruse List::Util 'sum'; 86*6236dae4SAndroid Build Coastguard Workeruse I18N::Langinfo qw(langinfo CODESET); 87*6236dae4SAndroid Build Coastguard Worker 88*6236dae4SAndroid Build Coastguard Workeruse pathhelp qw( 89*6236dae4SAndroid Build Coastguard Worker exe_ext 90*6236dae4SAndroid Build Coastguard Worker sys_native_current_path 91*6236dae4SAndroid Build Coastguard Worker ); 92*6236dae4SAndroid Build Coastguard Workeruse processhelp qw( 93*6236dae4SAndroid Build Coastguard Worker portable_sleep 94*6236dae4SAndroid Build Coastguard Worker ); 95*6236dae4SAndroid Build Coastguard Worker 96*6236dae4SAndroid Build Coastguard Workeruse appveyor; 97*6236dae4SAndroid Build Coastguard Workeruse azure; 98*6236dae4SAndroid Build Coastguard Workeruse getpart; # array functions 99*6236dae4SAndroid Build Coastguard Workeruse servers; 100*6236dae4SAndroid Build Coastguard Workeruse valgrind; # valgrind report parser 101*6236dae4SAndroid Build Coastguard Workeruse globalconfig; 102*6236dae4SAndroid Build Coastguard Workeruse runner; 103*6236dae4SAndroid Build Coastguard Workeruse testutil; 104*6236dae4SAndroid Build Coastguard Worker 105*6236dae4SAndroid Build Coastguard Workermy %custom_skip_reasons; 106*6236dae4SAndroid Build Coastguard Worker 107*6236dae4SAndroid Build Coastguard Workermy $ACURL=$VCURL; # what curl binary to use to talk to APIs (relevant for CI) 108*6236dae4SAndroid Build Coastguard Worker # ACURL is handy to set to the system one for reliability 109*6236dae4SAndroid Build Coastguard Workermy $CURLCONFIG="../curl-config"; # curl-config from current build 110*6236dae4SAndroid Build Coastguard Worker 111*6236dae4SAndroid Build Coastguard Worker# Normally, all test cases should be run, but at times it is handy to 112*6236dae4SAndroid Build Coastguard Worker# simply run a particular one: 113*6236dae4SAndroid Build Coastguard Workermy $TESTCASES="all"; 114*6236dae4SAndroid Build Coastguard Worker 115*6236dae4SAndroid Build Coastguard Worker# To run specific test cases, set them like: 116*6236dae4SAndroid Build Coastguard Worker# $TESTCASES="1 2 3 7 8"; 117*6236dae4SAndroid Build Coastguard Worker 118*6236dae4SAndroid Build Coastguard Worker####################################################################### 119*6236dae4SAndroid Build Coastguard Worker# No variables below this point should need to be modified 120*6236dae4SAndroid Build Coastguard Worker# 121*6236dae4SAndroid Build Coastguard Worker 122*6236dae4SAndroid Build Coastguard Workermy $libtool; 123*6236dae4SAndroid Build Coastguard Workermy $repeat = 0; 124*6236dae4SAndroid Build Coastguard Worker 125*6236dae4SAndroid Build Coastguard Workermy $start; # time at which testing started 126*6236dae4SAndroid Build Coastguard Workermy $args; # command-line arguments 127*6236dae4SAndroid Build Coastguard Worker 128*6236dae4SAndroid Build Coastguard Workermy $uname_release = `uname -r`; 129*6236dae4SAndroid Build Coastguard Workermy $is_wsl = $uname_release =~ /Microsoft$/; 130*6236dae4SAndroid Build Coastguard Worker 131*6236dae4SAndroid Build Coastguard Workermy $http_ipv6; # set if HTTP server has IPv6 support 132*6236dae4SAndroid Build Coastguard Workermy $http_unix; # set if HTTP server has Unix sockets support 133*6236dae4SAndroid Build Coastguard Workermy $ftp_ipv6; # set if FTP server has IPv6 support 134*6236dae4SAndroid Build Coastguard Worker 135*6236dae4SAndroid Build Coastguard Workermy $resolver; # name of the resolver backend (for human presentation) 136*6236dae4SAndroid Build Coastguard Worker 137*6236dae4SAndroid Build Coastguard Workermy %skipped; # skipped{reason}=counter, reasons for skip 138*6236dae4SAndroid Build Coastguard Workermy @teststat; # teststat[testnum]=reason, reasons for skip 139*6236dae4SAndroid Build Coastguard Workermy %disabled_keywords; # key words of tests to skip 140*6236dae4SAndroid Build Coastguard Workermy %ignored_keywords; # key words of tests to ignore results 141*6236dae4SAndroid Build Coastguard Workermy %enabled_keywords; # key words of tests to run 142*6236dae4SAndroid Build Coastguard Workermy %disabled; # disabled test cases 143*6236dae4SAndroid Build Coastguard Workermy %ignored; # ignored results of test cases 144*6236dae4SAndroid Build Coastguard Workermy %ignoretestcodes; # if test results are to be ignored 145*6236dae4SAndroid Build Coastguard Worker 146*6236dae4SAndroid Build Coastguard Workermy $passedign; # tests passed with results ignored 147*6236dae4SAndroid Build Coastguard Worker 148*6236dae4SAndroid Build Coastguard Workermy $timestats; # time stamping and stats generation 149*6236dae4SAndroid Build Coastguard Workermy $fullstats; # show time stats for every single test 150*6236dae4SAndroid Build Coastguard Workermy %timeprepini; # timestamp for each test preparation start 151*6236dae4SAndroid Build Coastguard Workermy %timesrvrini; # timestamp for each test required servers verification start 152*6236dae4SAndroid Build Coastguard Workermy %timesrvrend; # timestamp for each test required servers verification end 153*6236dae4SAndroid Build Coastguard Workermy %timetoolini; # timestamp for each test command run starting 154*6236dae4SAndroid Build Coastguard Workermy %timetoolend; # timestamp for each test command run stopping 155*6236dae4SAndroid Build Coastguard Workermy %timesrvrlog; # timestamp for each test server logs lock removal 156*6236dae4SAndroid Build Coastguard Workermy %timevrfyend; # timestamp for each test result verification end 157*6236dae4SAndroid Build Coastguard Workermy $globalabort; # flag signalling program abort 158*6236dae4SAndroid Build Coastguard Worker 159*6236dae4SAndroid Build Coastguard Worker# values for $singletest_state 160*6236dae4SAndroid Build Coastguard Workeruse constant { 161*6236dae4SAndroid Build Coastguard Worker ST_INIT => 0, 162*6236dae4SAndroid Build Coastguard Worker ST_CLEARLOCKS => 1, 163*6236dae4SAndroid Build Coastguard Worker ST_INITED => 2, 164*6236dae4SAndroid Build Coastguard Worker ST_PREPROCESS => 3, 165*6236dae4SAndroid Build Coastguard Worker ST_RUN => 4, 166*6236dae4SAndroid Build Coastguard Worker}; 167*6236dae4SAndroid Build Coastguard Workermy %singletest_state; # current state of singletest() by runner ID 168*6236dae4SAndroid Build Coastguard Workermy %singletest_logs; # log messages while in singletest array ref by runner 169*6236dae4SAndroid Build Coastguard Workermy $singletest_bufferedrunner; # runner ID which is buffering logs 170*6236dae4SAndroid Build Coastguard Workermy %runnerids; # runner IDs by number 171*6236dae4SAndroid Build Coastguard Workermy @runnersidle; # runner IDs idle and ready to execute a test 172*6236dae4SAndroid Build Coastguard Workermy %countforrunner; # test count by runner ID 173*6236dae4SAndroid Build Coastguard Workermy %runnersrunning; # tests currently running by runner ID 174*6236dae4SAndroid Build Coastguard Worker 175*6236dae4SAndroid Build Coastguard Worker####################################################################### 176*6236dae4SAndroid Build Coastguard Worker# variables that command line options may set 177*6236dae4SAndroid Build Coastguard Worker# 178*6236dae4SAndroid Build Coastguard Workermy $short; 179*6236dae4SAndroid Build Coastguard Workermy $no_debuginfod; 180*6236dae4SAndroid Build Coastguard Workermy $keepoutfiles; # keep stdout and stderr files after tests 181*6236dae4SAndroid Build Coastguard Workermy $clearlocks; # force removal of files by killing locking processes 182*6236dae4SAndroid Build Coastguard Workermy $postmortem; # display detailed info about failed tests 183*6236dae4SAndroid Build Coastguard Workermy $run_disabled; # run the specific tests even if listed in DISABLED 184*6236dae4SAndroid Build Coastguard Workermy $scrambleorder; 185*6236dae4SAndroid Build Coastguard Workermy $jobs = 0; 186*6236dae4SAndroid Build Coastguard Worker 187*6236dae4SAndroid Build Coastguard Worker# Azure Pipelines specific variables 188*6236dae4SAndroid Build Coastguard Workermy $AZURE_RUN_ID = 0; 189*6236dae4SAndroid Build Coastguard Workermy $AZURE_RESULT_ID = 0; 190*6236dae4SAndroid Build Coastguard Worker 191*6236dae4SAndroid Build Coastguard Worker####################################################################### 192*6236dae4SAndroid Build Coastguard Worker# logmsg is our general message logging subroutine. 193*6236dae4SAndroid Build Coastguard Worker# 194*6236dae4SAndroid Build Coastguard Workersub logmsg { 195*6236dae4SAndroid Build Coastguard Worker if($singletest_bufferedrunner) { 196*6236dae4SAndroid Build Coastguard Worker # Logs are currently being buffered 197*6236dae4SAndroid Build Coastguard Worker return singletest_logmsg(@_); 198*6236dae4SAndroid Build Coastguard Worker } 199*6236dae4SAndroid Build Coastguard Worker for(@_) { 200*6236dae4SAndroid Build Coastguard Worker my $line = $_; 201*6236dae4SAndroid Build Coastguard Worker if(!$line) { 202*6236dae4SAndroid Build Coastguard Worker next; 203*6236dae4SAndroid Build Coastguard Worker } 204*6236dae4SAndroid Build Coastguard Worker if ($is_wsl) { 205*6236dae4SAndroid Build Coastguard Worker # use \r\n for WSL shell 206*6236dae4SAndroid Build Coastguard Worker $line =~ s/\r?\n$/\r\n/g; 207*6236dae4SAndroid Build Coastguard Worker } 208*6236dae4SAndroid Build Coastguard Worker print "$line"; 209*6236dae4SAndroid Build Coastguard Worker } 210*6236dae4SAndroid Build Coastguard Worker} 211*6236dae4SAndroid Build Coastguard Worker 212*6236dae4SAndroid Build Coastguard Worker####################################################################### 213*6236dae4SAndroid Build Coastguard Worker# enable logmsg buffering for the given runner ID 214*6236dae4SAndroid Build Coastguard Worker# 215*6236dae4SAndroid Build Coastguard Workersub logmsg_bufferfortest { 216*6236dae4SAndroid Build Coastguard Worker my ($runnerid)=@_; 217*6236dae4SAndroid Build Coastguard Worker if($jobs) { 218*6236dae4SAndroid Build Coastguard Worker # Only enable buffering in multiprocess mode 219*6236dae4SAndroid Build Coastguard Worker $singletest_bufferedrunner = $runnerid; 220*6236dae4SAndroid Build Coastguard Worker } 221*6236dae4SAndroid Build Coastguard Worker} 222*6236dae4SAndroid Build Coastguard Worker####################################################################### 223*6236dae4SAndroid Build Coastguard Worker# Store a log message in a buffer for this test 224*6236dae4SAndroid Build Coastguard Worker# The messages can then be displayed all at once at the end of the test 225*6236dae4SAndroid Build Coastguard Worker# which prevents messages from different tests from being interleaved. 226*6236dae4SAndroid Build Coastguard Workersub singletest_logmsg { 227*6236dae4SAndroid Build Coastguard Worker if(!exists $singletest_logs{$singletest_bufferedrunner}) { 228*6236dae4SAndroid Build Coastguard Worker # initialize to a reference to an empty anonymous array 229*6236dae4SAndroid Build Coastguard Worker $singletest_logs{$singletest_bufferedrunner} = []; 230*6236dae4SAndroid Build Coastguard Worker } 231*6236dae4SAndroid Build Coastguard Worker my $logsref = $singletest_logs{$singletest_bufferedrunner}; 232*6236dae4SAndroid Build Coastguard Worker push @$logsref, @_; 233*6236dae4SAndroid Build Coastguard Worker} 234*6236dae4SAndroid Build Coastguard Worker 235*6236dae4SAndroid Build Coastguard Worker####################################################################### 236*6236dae4SAndroid Build Coastguard Worker# Stop buffering log messages, but don't touch them 237*6236dae4SAndroid Build Coastguard Workersub singletest_unbufferlogs { 238*6236dae4SAndroid Build Coastguard Worker undef $singletest_bufferedrunner; 239*6236dae4SAndroid Build Coastguard Worker} 240*6236dae4SAndroid Build Coastguard Worker 241*6236dae4SAndroid Build Coastguard Worker####################################################################### 242*6236dae4SAndroid Build Coastguard Worker# Clear the buffered log messages & stop buffering after returning them 243*6236dae4SAndroid Build Coastguard Workersub singletest_dumplogs { 244*6236dae4SAndroid Build Coastguard Worker if(!defined $singletest_bufferedrunner) { 245*6236dae4SAndroid Build Coastguard Worker # probably not multiprocess mode and logs weren't buffered 246*6236dae4SAndroid Build Coastguard Worker return undef; 247*6236dae4SAndroid Build Coastguard Worker } 248*6236dae4SAndroid Build Coastguard Worker my $logsref = $singletest_logs{$singletest_bufferedrunner}; 249*6236dae4SAndroid Build Coastguard Worker my $msg = join("", @$logsref); 250*6236dae4SAndroid Build Coastguard Worker delete $singletest_logs{$singletest_bufferedrunner}; 251*6236dae4SAndroid Build Coastguard Worker singletest_unbufferlogs(); 252*6236dae4SAndroid Build Coastguard Worker return $msg; 253*6236dae4SAndroid Build Coastguard Worker} 254*6236dae4SAndroid Build Coastguard Worker 255*6236dae4SAndroid Build Coastguard Workersub catch_zap { 256*6236dae4SAndroid Build Coastguard Worker my $signame = shift; 257*6236dae4SAndroid Build Coastguard Worker print "runtests.pl received SIG$signame, exiting\r\n"; 258*6236dae4SAndroid Build Coastguard Worker $globalabort = 1; 259*6236dae4SAndroid Build Coastguard Worker} 260*6236dae4SAndroid Build Coastguard Worker$SIG{INT} = \&catch_zap; 261*6236dae4SAndroid Build Coastguard Worker$SIG{TERM} = \&catch_zap; 262*6236dae4SAndroid Build Coastguard Worker 263*6236dae4SAndroid Build Coastguard Workersub catch_usr1 { 264*6236dae4SAndroid Build Coastguard Worker print "runtests.pl internal state:\r\n"; 265*6236dae4SAndroid Build Coastguard Worker print scalar(%runnersrunning) . " busy test runner(s) of " . scalar(keys %runnerids) . "\r\n"; 266*6236dae4SAndroid Build Coastguard Worker foreach my $rid (sort(keys(%runnersrunning))) { 267*6236dae4SAndroid Build Coastguard Worker my $runnernum = "unknown"; 268*6236dae4SAndroid Build Coastguard Worker foreach my $rnum (keys %runnerids) { 269*6236dae4SAndroid Build Coastguard Worker if($runnerids{$rnum} == $rid) { 270*6236dae4SAndroid Build Coastguard Worker $runnernum = $rnum; 271*6236dae4SAndroid Build Coastguard Worker last; 272*6236dae4SAndroid Build Coastguard Worker } 273*6236dae4SAndroid Build Coastguard Worker } 274*6236dae4SAndroid Build Coastguard Worker print "Runner $runnernum (id $rid) running test $runnersrunning{$rid} in state $singletest_state{$rid}\r\n"; 275*6236dae4SAndroid Build Coastguard Worker } 276*6236dae4SAndroid Build Coastguard Worker} 277*6236dae4SAndroid Build Coastguard Worker 278*6236dae4SAndroid Build Coastguard Workereval { 279*6236dae4SAndroid Build Coastguard Worker # some msys2 perl versions don't define SIGUSR1 280*6236dae4SAndroid Build Coastguard Worker $SIG{USR1} = \&catch_usr1; 281*6236dae4SAndroid Build Coastguard Worker}; 282*6236dae4SAndroid Build Coastguard Worker$SIG{PIPE} = 'IGNORE'; # these errors are captured in the read/write calls 283*6236dae4SAndroid Build Coastguard Worker 284*6236dae4SAndroid Build Coastguard Worker########################################################################## 285*6236dae4SAndroid Build Coastguard Worker# Clear all possible '*_proxy' environment variables for various protocols 286*6236dae4SAndroid Build Coastguard Worker# to prevent them to interfere with our testing! 287*6236dae4SAndroid Build Coastguard Worker 288*6236dae4SAndroid Build Coastguard Workerforeach my $protocol (('ftp', 'http', 'ftps', 'https', 'no', 'all')) { 289*6236dae4SAndroid Build Coastguard Worker my $proxy = "${protocol}_proxy"; 290*6236dae4SAndroid Build Coastguard Worker # clear lowercase version 291*6236dae4SAndroid Build Coastguard Worker delete $ENV{$proxy} if($ENV{$proxy}); 292*6236dae4SAndroid Build Coastguard Worker # clear uppercase version 293*6236dae4SAndroid Build Coastguard Worker delete $ENV{uc($proxy)} if($ENV{uc($proxy)}); 294*6236dae4SAndroid Build Coastguard Worker} 295*6236dae4SAndroid Build Coastguard Worker 296*6236dae4SAndroid Build Coastguard Worker# make sure we don't get affected by other variables that control our 297*6236dae4SAndroid Build Coastguard Worker# behavior 298*6236dae4SAndroid Build Coastguard Worker 299*6236dae4SAndroid Build Coastguard Workerdelete $ENV{'SSL_CERT_DIR'} if($ENV{'SSL_CERT_DIR'}); 300*6236dae4SAndroid Build Coastguard Workerdelete $ENV{'SSL_CERT_PATH'} if($ENV{'SSL_CERT_PATH'}); 301*6236dae4SAndroid Build Coastguard Workerdelete $ENV{'CURL_CA_BUNDLE'} if($ENV{'CURL_CA_BUNDLE'}); 302*6236dae4SAndroid Build Coastguard Worker 303*6236dae4SAndroid Build Coastguard Worker# provide defaults from our config file for ENV vars not explicitly 304*6236dae4SAndroid Build Coastguard Worker# set by the caller 305*6236dae4SAndroid Build Coastguard Workerif (open(my $fd, "<", "config")) { 306*6236dae4SAndroid Build Coastguard Worker while(my $line = <$fd>) { 307*6236dae4SAndroid Build Coastguard Worker next if ($line =~ /^#/); 308*6236dae4SAndroid Build Coastguard Worker chomp $line; 309*6236dae4SAndroid Build Coastguard Worker my ($name, $val) = split(/\s*:\s*/, $line, 2); 310*6236dae4SAndroid Build Coastguard Worker $ENV{$name} = $val if(!$ENV{$name}); 311*6236dae4SAndroid Build Coastguard Worker } 312*6236dae4SAndroid Build Coastguard Worker close($fd); 313*6236dae4SAndroid Build Coastguard Worker} 314*6236dae4SAndroid Build Coastguard Worker 315*6236dae4SAndroid Build Coastguard Worker# Check if we have nghttpx available and if it talks http/3 316*6236dae4SAndroid Build Coastguard Workermy $nghttpx_h3 = 0; 317*6236dae4SAndroid Build Coastguard Workerif (!$ENV{"NGHTTPX"}) { 318*6236dae4SAndroid Build Coastguard Worker $ENV{"NGHTTPX"} = checktestcmd("nghttpx"); 319*6236dae4SAndroid Build Coastguard Worker} 320*6236dae4SAndroid Build Coastguard Workerif ($ENV{"NGHTTPX"}) { 321*6236dae4SAndroid Build Coastguard Worker my $cmd = "\"$ENV{'NGHTTPX'}\" -v 2>$dev_null"; 322*6236dae4SAndroid Build Coastguard Worker my $nghttpx_version=join(' ', `$cmd`); 323*6236dae4SAndroid Build Coastguard Worker $nghttpx_h3 = $nghttpx_version =~ /nghttp3\//; 324*6236dae4SAndroid Build Coastguard Worker chomp $nghttpx_h3; 325*6236dae4SAndroid Build Coastguard Worker} 326*6236dae4SAndroid Build Coastguard Worker 327*6236dae4SAndroid Build Coastguard Worker 328*6236dae4SAndroid Build Coastguard Worker####################################################################### 329*6236dae4SAndroid Build Coastguard Worker# Get the list of tests that the tests/data/Makefile.am knows about! 330*6236dae4SAndroid Build Coastguard Worker# 331*6236dae4SAndroid Build Coastguard Workermy $disttests = ""; 332*6236dae4SAndroid Build Coastguard Workersub get_disttests { 333*6236dae4SAndroid Build Coastguard Worker # If a non-default $TESTDIR is being used there may not be any 334*6236dae4SAndroid Build Coastguard Worker # Makefile.am in which case there's nothing to do. 335*6236dae4SAndroid Build Coastguard Worker open(my $dh, "<", "$TESTDIR/Makefile.am") or return; 336*6236dae4SAndroid Build Coastguard Worker while(<$dh>) { 337*6236dae4SAndroid Build Coastguard Worker chomp $_; 338*6236dae4SAndroid Build Coastguard Worker if(($_ =~ /^#/) ||($_ !~ /test/)) { 339*6236dae4SAndroid Build Coastguard Worker next; 340*6236dae4SAndroid Build Coastguard Worker } 341*6236dae4SAndroid Build Coastguard Worker $disttests .= $_; 342*6236dae4SAndroid Build Coastguard Worker } 343*6236dae4SAndroid Build Coastguard Worker close($dh); 344*6236dae4SAndroid Build Coastguard Worker} 345*6236dae4SAndroid Build Coastguard Worker 346*6236dae4SAndroid Build Coastguard Worker 347*6236dae4SAndroid Build Coastguard Worker####################################################################### 348*6236dae4SAndroid Build Coastguard Worker# Remove all files in the specified directory 349*6236dae4SAndroid Build Coastguard Worker# 350*6236dae4SAndroid Build Coastguard Workersub cleardir { 351*6236dae4SAndroid Build Coastguard Worker my $dir = $_[0]; 352*6236dae4SAndroid Build Coastguard Worker my $done = 1; # success 353*6236dae4SAndroid Build Coastguard Worker my $file; 354*6236dae4SAndroid Build Coastguard Worker 355*6236dae4SAndroid Build Coastguard Worker # Get all files 356*6236dae4SAndroid Build Coastguard Worker opendir(my $dh, $dir) || 357*6236dae4SAndroid Build Coastguard Worker return 0; # can't open dir 358*6236dae4SAndroid Build Coastguard Worker while($file = readdir($dh)) { 359*6236dae4SAndroid Build Coastguard Worker # Don't clear the $PIDDIR or $LOCKDIR since those need to live beyond 360*6236dae4SAndroid Build Coastguard Worker # one test 361*6236dae4SAndroid Build Coastguard Worker if(($file !~ /^(\.|\.\.)\z/) && 362*6236dae4SAndroid Build Coastguard Worker "$file" ne $PIDDIR && "$file" ne $LOCKDIR) { 363*6236dae4SAndroid Build Coastguard Worker if(-d "$dir/$file") { 364*6236dae4SAndroid Build Coastguard Worker if(!cleardir("$dir/$file")) { 365*6236dae4SAndroid Build Coastguard Worker $done = 0; 366*6236dae4SAndroid Build Coastguard Worker } 367*6236dae4SAndroid Build Coastguard Worker if(!rmdir("$dir/$file")) { 368*6236dae4SAndroid Build Coastguard Worker $done = 0; 369*6236dae4SAndroid Build Coastguard Worker } 370*6236dae4SAndroid Build Coastguard Worker } 371*6236dae4SAndroid Build Coastguard Worker else { 372*6236dae4SAndroid Build Coastguard Worker # Ignore stunnel since we cannot do anything about its locks 373*6236dae4SAndroid Build Coastguard Worker if(!unlink("$dir/$file") && "$file" !~ /_stunnel\.log$/) { 374*6236dae4SAndroid Build Coastguard Worker $done = 0; 375*6236dae4SAndroid Build Coastguard Worker } 376*6236dae4SAndroid Build Coastguard Worker } 377*6236dae4SAndroid Build Coastguard Worker } 378*6236dae4SAndroid Build Coastguard Worker } 379*6236dae4SAndroid Build Coastguard Worker closedir $dh; 380*6236dae4SAndroid Build Coastguard Worker return $done; 381*6236dae4SAndroid Build Coastguard Worker} 382*6236dae4SAndroid Build Coastguard Worker 383*6236dae4SAndroid Build Coastguard Worker 384*6236dae4SAndroid Build Coastguard Worker####################################################################### 385*6236dae4SAndroid Build Coastguard Worker# Given two array references, this function will store them in two temporary 386*6236dae4SAndroid Build Coastguard Worker# files, run 'diff' on them, store the result and return the diff output! 387*6236dae4SAndroid Build Coastguard Workersub showdiff { 388*6236dae4SAndroid Build Coastguard Worker my ($logdir, $firstref, $secondref)=@_; 389*6236dae4SAndroid Build Coastguard Worker 390*6236dae4SAndroid Build Coastguard Worker my $file1="$logdir/check-generated"; 391*6236dae4SAndroid Build Coastguard Worker my $file2="$logdir/check-expected"; 392*6236dae4SAndroid Build Coastguard Worker 393*6236dae4SAndroid Build Coastguard Worker open(my $temp, ">", "$file1") || die "Failure writing diff file"; 394*6236dae4SAndroid Build Coastguard Worker for(@$firstref) { 395*6236dae4SAndroid Build Coastguard Worker my $l = $_; 396*6236dae4SAndroid Build Coastguard Worker $l =~ s/\r/[CR]/g; 397*6236dae4SAndroid Build Coastguard Worker $l =~ s/\n/[LF]/g; 398*6236dae4SAndroid Build Coastguard Worker $l =~ s/([^\x20-\x7f])/sprintf "%%%02x", ord $1/eg; 399*6236dae4SAndroid Build Coastguard Worker print $temp $l; 400*6236dae4SAndroid Build Coastguard Worker print $temp "\n"; 401*6236dae4SAndroid Build Coastguard Worker } 402*6236dae4SAndroid Build Coastguard Worker close($temp) || die "Failure writing diff file"; 403*6236dae4SAndroid Build Coastguard Worker 404*6236dae4SAndroid Build Coastguard Worker open($temp, ">", "$file2") || die "Failure writing diff file"; 405*6236dae4SAndroid Build Coastguard Worker for(@$secondref) { 406*6236dae4SAndroid Build Coastguard Worker my $l = $_; 407*6236dae4SAndroid Build Coastguard Worker $l =~ s/\r/[CR]/g; 408*6236dae4SAndroid Build Coastguard Worker $l =~ s/\n/[LF]/g; 409*6236dae4SAndroid Build Coastguard Worker $l =~ s/([^\x20-\x7f])/sprintf "%%%02x", ord $1/eg; 410*6236dae4SAndroid Build Coastguard Worker print $temp $l; 411*6236dae4SAndroid Build Coastguard Worker print $temp "\n"; 412*6236dae4SAndroid Build Coastguard Worker } 413*6236dae4SAndroid Build Coastguard Worker close($temp) || die "Failure writing diff file"; 414*6236dae4SAndroid Build Coastguard Worker my @out = `diff -u $file2 $file1 2>$dev_null`; 415*6236dae4SAndroid Build Coastguard Worker 416*6236dae4SAndroid Build Coastguard Worker if(!$out[0]) { 417*6236dae4SAndroid Build Coastguard Worker @out = `diff -c $file2 $file1 2>$dev_null`; 418*6236dae4SAndroid Build Coastguard Worker } 419*6236dae4SAndroid Build Coastguard Worker 420*6236dae4SAndroid Build Coastguard Worker return @out; 421*6236dae4SAndroid Build Coastguard Worker} 422*6236dae4SAndroid Build Coastguard Worker 423*6236dae4SAndroid Build Coastguard Worker 424*6236dae4SAndroid Build Coastguard Worker####################################################################### 425*6236dae4SAndroid Build Coastguard Worker# compare test results with the expected output, we might filter off 426*6236dae4SAndroid Build Coastguard Worker# some pattern that is allowed to differ, output test results 427*6236dae4SAndroid Build Coastguard Worker# 428*6236dae4SAndroid Build Coastguard Workersub compare { 429*6236dae4SAndroid Build Coastguard Worker my ($runnerid, $testnum, $testname, $subject, $firstref, $secondref)=@_; 430*6236dae4SAndroid Build Coastguard Worker 431*6236dae4SAndroid Build Coastguard Worker my $result = compareparts($firstref, $secondref); 432*6236dae4SAndroid Build Coastguard Worker 433*6236dae4SAndroid Build Coastguard Worker if($result) { 434*6236dae4SAndroid Build Coastguard Worker # timestamp test result verification end 435*6236dae4SAndroid Build Coastguard Worker $timevrfyend{$testnum} = Time::HiRes::time(); 436*6236dae4SAndroid Build Coastguard Worker 437*6236dae4SAndroid Build Coastguard Worker if(!$short) { 438*6236dae4SAndroid Build Coastguard Worker logmsg "\n $testnum: $subject FAILED:\n"; 439*6236dae4SAndroid Build Coastguard Worker my $logdir = getrunnerlogdir($runnerid); 440*6236dae4SAndroid Build Coastguard Worker logmsg showdiff($logdir, $firstref, $secondref); 441*6236dae4SAndroid Build Coastguard Worker } 442*6236dae4SAndroid Build Coastguard Worker elsif(!$automakestyle) { 443*6236dae4SAndroid Build Coastguard Worker logmsg "FAILED\n"; 444*6236dae4SAndroid Build Coastguard Worker } 445*6236dae4SAndroid Build Coastguard Worker else { 446*6236dae4SAndroid Build Coastguard Worker # automakestyle 447*6236dae4SAndroid Build Coastguard Worker logmsg "FAIL: $testnum - $testname - $subject\n"; 448*6236dae4SAndroid Build Coastguard Worker } 449*6236dae4SAndroid Build Coastguard Worker } 450*6236dae4SAndroid Build Coastguard Worker return $result; 451*6236dae4SAndroid Build Coastguard Worker} 452*6236dae4SAndroid Build Coastguard Worker 453*6236dae4SAndroid Build Coastguard Worker####################################################################### 454*6236dae4SAndroid Build Coastguard Worker# Numeric-sort words in a string 455*6236dae4SAndroid Build Coastguard Workersub numsortwords { 456*6236dae4SAndroid Build Coastguard Worker my ($string)=@_; 457*6236dae4SAndroid Build Coastguard Worker return join(' ', sort { $a <=> $b } split(' ', $string)); 458*6236dae4SAndroid Build Coastguard Worker} 459*6236dae4SAndroid Build Coastguard Worker 460*6236dae4SAndroid Build Coastguard Worker####################################################################### 461*6236dae4SAndroid Build Coastguard Worker# Parse and store the protocols in curl's Protocols: line 462*6236dae4SAndroid Build Coastguard Workersub parseprotocols { 463*6236dae4SAndroid Build Coastguard Worker my ($line)=@_; 464*6236dae4SAndroid Build Coastguard Worker 465*6236dae4SAndroid Build Coastguard Worker @protocols = split(' ', lc($line)); 466*6236dae4SAndroid Build Coastguard Worker 467*6236dae4SAndroid Build Coastguard Worker # Generate a "proto-ipv6" version of each protocol to match the 468*6236dae4SAndroid Build Coastguard Worker # IPv6 <server> name and a "proto-unix" to match the variant which 469*6236dae4SAndroid Build Coastguard Worker # uses Unix domain sockets. This works even if support isn't 470*6236dae4SAndroid Build Coastguard Worker # compiled in because the <features> test will fail. 471*6236dae4SAndroid Build Coastguard Worker push @protocols, map(("$_-ipv6", "$_-unix"), @protocols); 472*6236dae4SAndroid Build Coastguard Worker 473*6236dae4SAndroid Build Coastguard Worker # 'http-proxy' is used in test cases to do CONNECT through 474*6236dae4SAndroid Build Coastguard Worker push @protocols, 'http-proxy'; 475*6236dae4SAndroid Build Coastguard Worker 476*6236dae4SAndroid Build Coastguard Worker # 'none' is used in test cases to mean no server 477*6236dae4SAndroid Build Coastguard Worker push @protocols, 'none'; 478*6236dae4SAndroid Build Coastguard Worker} 479*6236dae4SAndroid Build Coastguard Worker 480*6236dae4SAndroid Build Coastguard Worker 481*6236dae4SAndroid Build Coastguard Worker####################################################################### 482*6236dae4SAndroid Build Coastguard Worker# Check & display information about curl and the host the test suite runs on. 483*6236dae4SAndroid Build Coastguard Worker# Information to do with servers is displayed in displayserverfeatures, after 484*6236dae4SAndroid Build Coastguard Worker# the server initialization is performed. 485*6236dae4SAndroid Build Coastguard Workersub checksystemfeatures { 486*6236dae4SAndroid Build Coastguard Worker my $proto; 487*6236dae4SAndroid Build Coastguard Worker my $feat; 488*6236dae4SAndroid Build Coastguard Worker my $curl; 489*6236dae4SAndroid Build Coastguard Worker my $libcurl; 490*6236dae4SAndroid Build Coastguard Worker my $versretval; 491*6236dae4SAndroid Build Coastguard Worker my $versnoexec; 492*6236dae4SAndroid Build Coastguard Worker my @version=(); 493*6236dae4SAndroid Build Coastguard Worker my @disabled; 494*6236dae4SAndroid Build Coastguard Worker my $dis = ""; 495*6236dae4SAndroid Build Coastguard Worker 496*6236dae4SAndroid Build Coastguard Worker my $curlverout="$LOGDIR/curlverout.log"; 497*6236dae4SAndroid Build Coastguard Worker my $curlvererr="$LOGDIR/curlvererr.log"; 498*6236dae4SAndroid Build Coastguard Worker my $versioncmd=shell_quote($CURL) . " --version 1>$curlverout 2>$curlvererr"; 499*6236dae4SAndroid Build Coastguard Worker 500*6236dae4SAndroid Build Coastguard Worker unlink($curlverout); 501*6236dae4SAndroid Build Coastguard Worker unlink($curlvererr); 502*6236dae4SAndroid Build Coastguard Worker 503*6236dae4SAndroid Build Coastguard Worker $versretval = runclient($versioncmd); 504*6236dae4SAndroid Build Coastguard Worker $versnoexec = $!; 505*6236dae4SAndroid Build Coastguard Worker 506*6236dae4SAndroid Build Coastguard Worker my $current_time = int(time()); 507*6236dae4SAndroid Build Coastguard Worker $ENV{'SOURCE_DATE_EPOCH'} = $current_time; 508*6236dae4SAndroid Build Coastguard Worker $DATE = strftime "%Y-%m-%d", gmtime($current_time); 509*6236dae4SAndroid Build Coastguard Worker 510*6236dae4SAndroid Build Coastguard Worker open(my $versout, "<", "$curlverout"); 511*6236dae4SAndroid Build Coastguard Worker @version = <$versout>; 512*6236dae4SAndroid Build Coastguard Worker close($versout); 513*6236dae4SAndroid Build Coastguard Worker 514*6236dae4SAndroid Build Coastguard Worker open(my $disabledh, "-|", "server/disabled".exe_ext('TOOL')); 515*6236dae4SAndroid Build Coastguard Worker @disabled = <$disabledh>; 516*6236dae4SAndroid Build Coastguard Worker close($disabledh); 517*6236dae4SAndroid Build Coastguard Worker 518*6236dae4SAndroid Build Coastguard Worker if($disabled[0]) { 519*6236dae4SAndroid Build Coastguard Worker s/[\r\n]//g for @disabled; 520*6236dae4SAndroid Build Coastguard Worker $dis = join(", ", @disabled); 521*6236dae4SAndroid Build Coastguard Worker } 522*6236dae4SAndroid Build Coastguard Worker 523*6236dae4SAndroid Build Coastguard Worker $resolver="stock"; 524*6236dae4SAndroid Build Coastguard Worker for(@version) { 525*6236dae4SAndroid Build Coastguard Worker chomp; 526*6236dae4SAndroid Build Coastguard Worker 527*6236dae4SAndroid Build Coastguard Worker if($_ =~ /^curl ([^ ]*)/) { 528*6236dae4SAndroid Build Coastguard Worker $curl = $_; 529*6236dae4SAndroid Build Coastguard Worker $CURLVERSION = $1; 530*6236dae4SAndroid Build Coastguard Worker $CURLVERNUM = $CURLVERSION; 531*6236dae4SAndroid Build Coastguard Worker $CURLVERNUM =~ s/^([0-9.]+)(.*)/$1/; # leading dots and numbers 532*6236dae4SAndroid Build Coastguard Worker $curl =~ s/^(.*)(libcurl.*)/$1/g || die "Failure determining curl binary version"; 533*6236dae4SAndroid Build Coastguard Worker 534*6236dae4SAndroid Build Coastguard Worker $libcurl = $2; 535*6236dae4SAndroid Build Coastguard Worker if($curl =~ /linux|bsd|solaris/) { 536*6236dae4SAndroid Build Coastguard Worker # system support LD_PRELOAD; may be disabled later 537*6236dae4SAndroid Build Coastguard Worker $feature{"ld_preload"} = 1; 538*6236dae4SAndroid Build Coastguard Worker } 539*6236dae4SAndroid Build Coastguard Worker if($curl =~ /win32|Windows|mingw(32|64)/) { 540*6236dae4SAndroid Build Coastguard Worker # This is a Windows MinGW build or native build, we need to use 541*6236dae4SAndroid Build Coastguard Worker # Windows-style path. 542*6236dae4SAndroid Build Coastguard Worker $pwd = sys_native_current_path(); 543*6236dae4SAndroid Build Coastguard Worker $feature{"win32"} = 1; 544*6236dae4SAndroid Build Coastguard Worker } 545*6236dae4SAndroid Build Coastguard Worker if ($libcurl =~ /\s(winssl|schannel)\b/i) { 546*6236dae4SAndroid Build Coastguard Worker $feature{"Schannel"} = 1; 547*6236dae4SAndroid Build Coastguard Worker $feature{"SSLpinning"} = 1; 548*6236dae4SAndroid Build Coastguard Worker } 549*6236dae4SAndroid Build Coastguard Worker elsif ($libcurl =~ /\sopenssl\b/i) { 550*6236dae4SAndroid Build Coastguard Worker $feature{"OpenSSL"} = 1; 551*6236dae4SAndroid Build Coastguard Worker $feature{"SSLpinning"} = 1; 552*6236dae4SAndroid Build Coastguard Worker } 553*6236dae4SAndroid Build Coastguard Worker elsif ($libcurl =~ /\sgnutls\b/i) { 554*6236dae4SAndroid Build Coastguard Worker $feature{"GnuTLS"} = 1; 555*6236dae4SAndroid Build Coastguard Worker $feature{"SSLpinning"} = 1; 556*6236dae4SAndroid Build Coastguard Worker } 557*6236dae4SAndroid Build Coastguard Worker elsif ($libcurl =~ /\srustls-ffi\b/i) { 558*6236dae4SAndroid Build Coastguard Worker $feature{"rustls"} = 1; 559*6236dae4SAndroid Build Coastguard Worker } 560*6236dae4SAndroid Build Coastguard Worker elsif ($libcurl =~ /\swolfssl\b/i) { 561*6236dae4SAndroid Build Coastguard Worker $feature{"wolfssl"} = 1; 562*6236dae4SAndroid Build Coastguard Worker $feature{"SSLpinning"} = 1; 563*6236dae4SAndroid Build Coastguard Worker } 564*6236dae4SAndroid Build Coastguard Worker elsif ($libcurl =~ /\sbearssl\b/i) { 565*6236dae4SAndroid Build Coastguard Worker $feature{"bearssl"} = 1; 566*6236dae4SAndroid Build Coastguard Worker } 567*6236dae4SAndroid Build Coastguard Worker elsif ($libcurl =~ /\ssecuretransport\b/i) { 568*6236dae4SAndroid Build Coastguard Worker $feature{"sectransp"} = 1; 569*6236dae4SAndroid Build Coastguard Worker $feature{"SSLpinning"} = 1; 570*6236dae4SAndroid Build Coastguard Worker } 571*6236dae4SAndroid Build Coastguard Worker elsif ($libcurl =~ /\sBoringSSL\b/i) { 572*6236dae4SAndroid Build Coastguard Worker # OpenSSL compatible API 573*6236dae4SAndroid Build Coastguard Worker $feature{"OpenSSL"} = 1; 574*6236dae4SAndroid Build Coastguard Worker $feature{"SSLpinning"} = 1; 575*6236dae4SAndroid Build Coastguard Worker } 576*6236dae4SAndroid Build Coastguard Worker elsif ($libcurl =~ /\slibressl\b/i) { 577*6236dae4SAndroid Build Coastguard Worker # OpenSSL compatible API 578*6236dae4SAndroid Build Coastguard Worker $feature{"OpenSSL"} = 1; 579*6236dae4SAndroid Build Coastguard Worker $feature{"SSLpinning"} = 1; 580*6236dae4SAndroid Build Coastguard Worker } 581*6236dae4SAndroid Build Coastguard Worker elsif ($libcurl =~ /\squictls\b/i) { 582*6236dae4SAndroid Build Coastguard Worker # OpenSSL compatible API 583*6236dae4SAndroid Build Coastguard Worker $feature{"OpenSSL"} = 1; 584*6236dae4SAndroid Build Coastguard Worker $feature{"SSLpinning"} = 1; 585*6236dae4SAndroid Build Coastguard Worker } 586*6236dae4SAndroid Build Coastguard Worker elsif ($libcurl =~ /\smbedTLS\b/i) { 587*6236dae4SAndroid Build Coastguard Worker $feature{"mbedtls"} = 1; 588*6236dae4SAndroid Build Coastguard Worker $feature{"SSLpinning"} = 1; 589*6236dae4SAndroid Build Coastguard Worker } 590*6236dae4SAndroid Build Coastguard Worker if ($libcurl =~ /ares/i) { 591*6236dae4SAndroid Build Coastguard Worker $feature{"c-ares"} = 1; 592*6236dae4SAndroid Build Coastguard Worker $resolver="c-ares"; 593*6236dae4SAndroid Build Coastguard Worker } 594*6236dae4SAndroid Build Coastguard Worker if ($libcurl =~ /Hyper/i) { 595*6236dae4SAndroid Build Coastguard Worker $feature{"hyper"} = 1; 596*6236dae4SAndroid Build Coastguard Worker } 597*6236dae4SAndroid Build Coastguard Worker if ($libcurl =~ /nghttp2/i) { 598*6236dae4SAndroid Build Coastguard Worker # nghttp2 supports h2c, hyper does not 599*6236dae4SAndroid Build Coastguard Worker $feature{"h2c"} = 1; 600*6236dae4SAndroid Build Coastguard Worker } 601*6236dae4SAndroid Build Coastguard Worker if ($libcurl =~ /AppleIDN/) { 602*6236dae4SAndroid Build Coastguard Worker $feature{"AppleIDN"} = 1; 603*6236dae4SAndroid Build Coastguard Worker } 604*6236dae4SAndroid Build Coastguard Worker if ($libcurl =~ /WinIDN/) { 605*6236dae4SAndroid Build Coastguard Worker $feature{"WinIDN"} = 1; 606*6236dae4SAndroid Build Coastguard Worker } 607*6236dae4SAndroid Build Coastguard Worker if ($libcurl =~ /libidn2/) { 608*6236dae4SAndroid Build Coastguard Worker $feature{"libidn2"} = 1; 609*6236dae4SAndroid Build Coastguard Worker } 610*6236dae4SAndroid Build Coastguard Worker if ($libcurl =~ /libssh2/i) { 611*6236dae4SAndroid Build Coastguard Worker $feature{"libssh2"} = 1; 612*6236dae4SAndroid Build Coastguard Worker } 613*6236dae4SAndroid Build Coastguard Worker if ($libcurl =~ /libssh\/([0-9.]*)\//i) { 614*6236dae4SAndroid Build Coastguard Worker $feature{"libssh"} = 1; 615*6236dae4SAndroid Build Coastguard Worker if($1 =~ /(\d+)\.(\d+).(\d+)/) { 616*6236dae4SAndroid Build Coastguard Worker my $v = $1 * 100 + $2 * 10 + $3; 617*6236dae4SAndroid Build Coastguard Worker if($v < 94) { 618*6236dae4SAndroid Build Coastguard Worker # before 0.9.4 619*6236dae4SAndroid Build Coastguard Worker $feature{"oldlibssh"} = 1; 620*6236dae4SAndroid Build Coastguard Worker } 621*6236dae4SAndroid Build Coastguard Worker } 622*6236dae4SAndroid Build Coastguard Worker } 623*6236dae4SAndroid Build Coastguard Worker if ($libcurl =~ /wolfssh/i) { 624*6236dae4SAndroid Build Coastguard Worker $feature{"wolfssh"} = 1; 625*6236dae4SAndroid Build Coastguard Worker } 626*6236dae4SAndroid Build Coastguard Worker } 627*6236dae4SAndroid Build Coastguard Worker elsif($_ =~ /^Protocols: (.*)/i) { 628*6236dae4SAndroid Build Coastguard Worker $proto = $1; 629*6236dae4SAndroid Build Coastguard Worker # these are the protocols compiled in to this libcurl 630*6236dae4SAndroid Build Coastguard Worker parseprotocols($proto); 631*6236dae4SAndroid Build Coastguard Worker } 632*6236dae4SAndroid Build Coastguard Worker elsif($_ =~ /^Features: (.*)/i) { 633*6236dae4SAndroid Build Coastguard Worker $feat = $1; 634*6236dae4SAndroid Build Coastguard Worker 635*6236dae4SAndroid Build Coastguard Worker # built with memory tracking support (--enable-curldebug); may be disabled later 636*6236dae4SAndroid Build Coastguard Worker $feature{"TrackMemory"} = $feat =~ /TrackMemory/i; 637*6236dae4SAndroid Build Coastguard Worker # curl was built with --enable-debug 638*6236dae4SAndroid Build Coastguard Worker $feature{"Debug"} = $feat =~ /Debug/i; 639*6236dae4SAndroid Build Coastguard Worker # ssl enabled 640*6236dae4SAndroid Build Coastguard Worker $feature{"SSL"} = $feat =~ /SSL/i; 641*6236dae4SAndroid Build Coastguard Worker # multiple ssl backends available. 642*6236dae4SAndroid Build Coastguard Worker $feature{"MultiSSL"} = $feat =~ /MultiSSL/i; 643*6236dae4SAndroid Build Coastguard Worker # large file support 644*6236dae4SAndroid Build Coastguard Worker $feature{"Largefile"} = $feat =~ /Largefile/i; 645*6236dae4SAndroid Build Coastguard Worker # IDN support 646*6236dae4SAndroid Build Coastguard Worker $feature{"IDN"} = $feat =~ /IDN/i; 647*6236dae4SAndroid Build Coastguard Worker # IPv6 support 648*6236dae4SAndroid Build Coastguard Worker $feature{"IPv6"} = $feat =~ /IPv6/i; 649*6236dae4SAndroid Build Coastguard Worker # Unix sockets support 650*6236dae4SAndroid Build Coastguard Worker $feature{"UnixSockets"} = $feat =~ /UnixSockets/i; 651*6236dae4SAndroid Build Coastguard Worker # libz compression 652*6236dae4SAndroid Build Coastguard Worker $feature{"libz"} = $feat =~ /libz/i; 653*6236dae4SAndroid Build Coastguard Worker # Brotli compression 654*6236dae4SAndroid Build Coastguard Worker $feature{"brotli"} = $feat =~ /brotli/i; 655*6236dae4SAndroid Build Coastguard Worker # Zstd compression 656*6236dae4SAndroid Build Coastguard Worker $feature{"zstd"} = $feat =~ /zstd/i; 657*6236dae4SAndroid Build Coastguard Worker # NTLM enabled 658*6236dae4SAndroid Build Coastguard Worker $feature{"NTLM"} = $feat =~ /NTLM/i; 659*6236dae4SAndroid Build Coastguard Worker # NTLM delegation to winbind daemon ntlm_auth helper enabled 660*6236dae4SAndroid Build Coastguard Worker $feature{"NTLM_WB"} = $feat =~ /NTLM_WB/i; 661*6236dae4SAndroid Build Coastguard Worker # SSPI enabled 662*6236dae4SAndroid Build Coastguard Worker $feature{"SSPI"} = $feat =~ /SSPI/i; 663*6236dae4SAndroid Build Coastguard Worker # GSS-API enabled 664*6236dae4SAndroid Build Coastguard Worker $feature{"GSS-API"} = $feat =~ /GSS-API/i; 665*6236dae4SAndroid Build Coastguard Worker # Kerberos enabled 666*6236dae4SAndroid Build Coastguard Worker $feature{"Kerberos"} = $feat =~ /Kerberos/i; 667*6236dae4SAndroid Build Coastguard Worker # SPNEGO enabled 668*6236dae4SAndroid Build Coastguard Worker $feature{"SPNEGO"} = $feat =~ /SPNEGO/i; 669*6236dae4SAndroid Build Coastguard Worker # TLS-SRP enabled 670*6236dae4SAndroid Build Coastguard Worker $feature{"TLS-SRP"} = $feat =~ /TLS-SRP/i; 671*6236dae4SAndroid Build Coastguard Worker # PSL enabled 672*6236dae4SAndroid Build Coastguard Worker $feature{"PSL"} = $feat =~ /PSL/i; 673*6236dae4SAndroid Build Coastguard Worker # alt-svc enabled 674*6236dae4SAndroid Build Coastguard Worker $feature{"alt-svc"} = $feat =~ /alt-svc/i; 675*6236dae4SAndroid Build Coastguard Worker # HSTS support 676*6236dae4SAndroid Build Coastguard Worker $feature{"HSTS"} = $feat =~ /HSTS/i; 677*6236dae4SAndroid Build Coastguard Worker if($feat =~ /AsynchDNS/i) { 678*6236dae4SAndroid Build Coastguard Worker if(!$feature{"c-ares"}) { 679*6236dae4SAndroid Build Coastguard Worker # this means threaded resolver 680*6236dae4SAndroid Build Coastguard Worker $feature{"threaded-resolver"} = 1; 681*6236dae4SAndroid Build Coastguard Worker $resolver="threaded"; 682*6236dae4SAndroid Build Coastguard Worker } 683*6236dae4SAndroid Build Coastguard Worker } 684*6236dae4SAndroid Build Coastguard Worker # http2 enabled 685*6236dae4SAndroid Build Coastguard Worker $feature{"http/2"} = $feat =~ /HTTP2/; 686*6236dae4SAndroid Build Coastguard Worker if($feature{"http/2"}) { 687*6236dae4SAndroid Build Coastguard Worker push @protocols, 'http/2'; 688*6236dae4SAndroid Build Coastguard Worker } 689*6236dae4SAndroid Build Coastguard Worker # http3 enabled 690*6236dae4SAndroid Build Coastguard Worker $feature{"http/3"} = $feat =~ /HTTP3/; 691*6236dae4SAndroid Build Coastguard Worker if($feature{"http/3"}) { 692*6236dae4SAndroid Build Coastguard Worker push @protocols, 'http/3'; 693*6236dae4SAndroid Build Coastguard Worker } 694*6236dae4SAndroid Build Coastguard Worker # https proxy support 695*6236dae4SAndroid Build Coastguard Worker $feature{"HTTPS-proxy"} = $feat =~ /HTTPS-proxy/; 696*6236dae4SAndroid Build Coastguard Worker if($feature{"HTTPS-proxy"}) { 697*6236dae4SAndroid Build Coastguard Worker # 'https-proxy' is used as "server" so consider it a protocol 698*6236dae4SAndroid Build Coastguard Worker push @protocols, 'https-proxy'; 699*6236dae4SAndroid Build Coastguard Worker } 700*6236dae4SAndroid Build Coastguard Worker # Unicode support 701*6236dae4SAndroid Build Coastguard Worker $feature{"Unicode"} = $feat =~ /Unicode/i; 702*6236dae4SAndroid Build Coastguard Worker # Thread-safe init 703*6236dae4SAndroid Build Coastguard Worker $feature{"threadsafe"} = $feat =~ /threadsafe/i; 704*6236dae4SAndroid Build Coastguard Worker } 705*6236dae4SAndroid Build Coastguard Worker # 706*6236dae4SAndroid Build Coastguard Worker # Test harness currently uses a non-stunnel server in order to 707*6236dae4SAndroid Build Coastguard Worker # run HTTP TLS-SRP tests required when curl is built with https 708*6236dae4SAndroid Build Coastguard Worker # protocol support and TLS-SRP feature enabled. For convenience 709*6236dae4SAndroid Build Coastguard Worker # 'httptls' may be included in the test harness protocols array 710*6236dae4SAndroid Build Coastguard Worker # to differentiate this from classic stunnel based 'https' test 711*6236dae4SAndroid Build Coastguard Worker # harness server. 712*6236dae4SAndroid Build Coastguard Worker # 713*6236dae4SAndroid Build Coastguard Worker if($feature{"TLS-SRP"}) { 714*6236dae4SAndroid Build Coastguard Worker my $add_httptls; 715*6236dae4SAndroid Build Coastguard Worker for(@protocols) { 716*6236dae4SAndroid Build Coastguard Worker if($_ =~ /^https(-ipv6|)$/) { 717*6236dae4SAndroid Build Coastguard Worker $add_httptls=1; 718*6236dae4SAndroid Build Coastguard Worker last; 719*6236dae4SAndroid Build Coastguard Worker } 720*6236dae4SAndroid Build Coastguard Worker } 721*6236dae4SAndroid Build Coastguard Worker if($add_httptls && (! grep /^httptls$/, @protocols)) { 722*6236dae4SAndroid Build Coastguard Worker push @protocols, 'httptls'; 723*6236dae4SAndroid Build Coastguard Worker push @protocols, 'httptls-ipv6'; 724*6236dae4SAndroid Build Coastguard Worker } 725*6236dae4SAndroid Build Coastguard Worker } 726*6236dae4SAndroid Build Coastguard Worker } 727*6236dae4SAndroid Build Coastguard Worker 728*6236dae4SAndroid Build Coastguard Worker if(!$curl) { 729*6236dae4SAndroid Build Coastguard Worker logmsg "unable to get curl's version, further details are:\n"; 730*6236dae4SAndroid Build Coastguard Worker logmsg "issued command: \n"; 731*6236dae4SAndroid Build Coastguard Worker logmsg "$versioncmd \n"; 732*6236dae4SAndroid Build Coastguard Worker if ($versretval == -1) { 733*6236dae4SAndroid Build Coastguard Worker logmsg "command failed with: \n"; 734*6236dae4SAndroid Build Coastguard Worker logmsg "$versnoexec \n"; 735*6236dae4SAndroid Build Coastguard Worker } 736*6236dae4SAndroid Build Coastguard Worker elsif ($versretval & 127) { 737*6236dae4SAndroid Build Coastguard Worker logmsg sprintf("command died with signal %d, and %s coredump.\n", 738*6236dae4SAndroid Build Coastguard Worker ($versretval & 127), ($versretval & 128)?"a":"no"); 739*6236dae4SAndroid Build Coastguard Worker } 740*6236dae4SAndroid Build Coastguard Worker else { 741*6236dae4SAndroid Build Coastguard Worker logmsg sprintf("command exited with value %d \n", $versretval >> 8); 742*6236dae4SAndroid Build Coastguard Worker } 743*6236dae4SAndroid Build Coastguard Worker logmsg "contents of $curlverout: \n"; 744*6236dae4SAndroid Build Coastguard Worker displaylogcontent("$curlverout"); 745*6236dae4SAndroid Build Coastguard Worker logmsg "contents of $curlvererr: \n"; 746*6236dae4SAndroid Build Coastguard Worker displaylogcontent("$curlvererr"); 747*6236dae4SAndroid Build Coastguard Worker die "couldn't get curl's version"; 748*6236dae4SAndroid Build Coastguard Worker } 749*6236dae4SAndroid Build Coastguard Worker 750*6236dae4SAndroid Build Coastguard Worker if(-r "../lib/curl_config.h") { 751*6236dae4SAndroid Build Coastguard Worker open(my $conf, "<", "../lib/curl_config.h"); 752*6236dae4SAndroid Build Coastguard Worker while(<$conf>) { 753*6236dae4SAndroid Build Coastguard Worker if($_ =~ /^\#define HAVE_GETRLIMIT/) { 754*6236dae4SAndroid Build Coastguard Worker # set if system has getrlimit() 755*6236dae4SAndroid Build Coastguard Worker $feature{"getrlimit"} = 1; 756*6236dae4SAndroid Build Coastguard Worker } 757*6236dae4SAndroid Build Coastguard Worker } 758*6236dae4SAndroid Build Coastguard Worker close($conf); 759*6236dae4SAndroid Build Coastguard Worker } 760*6236dae4SAndroid Build Coastguard Worker 761*6236dae4SAndroid Build Coastguard Worker # allow this feature only if debug mode is disabled 762*6236dae4SAndroid Build Coastguard Worker $feature{"ld_preload"} = $feature{"ld_preload"} && !$feature{"Debug"}; 763*6236dae4SAndroid Build Coastguard Worker 764*6236dae4SAndroid Build Coastguard Worker if($feature{"IPv6"}) { 765*6236dae4SAndroid Build Coastguard Worker # client has IPv6 support 766*6236dae4SAndroid Build Coastguard Worker 767*6236dae4SAndroid Build Coastguard Worker # check if the HTTP server has it! 768*6236dae4SAndroid Build Coastguard Worker my $cmd = "server/sws".exe_ext('SRV')." --version"; 769*6236dae4SAndroid Build Coastguard Worker my @sws = `$cmd`; 770*6236dae4SAndroid Build Coastguard Worker if($sws[0] =~ /IPv6/) { 771*6236dae4SAndroid Build Coastguard Worker # HTTP server has IPv6 support! 772*6236dae4SAndroid Build Coastguard Worker $http_ipv6 = 1; 773*6236dae4SAndroid Build Coastguard Worker } 774*6236dae4SAndroid Build Coastguard Worker 775*6236dae4SAndroid Build Coastguard Worker # check if the FTP server has it! 776*6236dae4SAndroid Build Coastguard Worker $cmd = "server/sockfilt".exe_ext('SRV')." --version"; 777*6236dae4SAndroid Build Coastguard Worker @sws = `$cmd`; 778*6236dae4SAndroid Build Coastguard Worker if($sws[0] =~ /IPv6/) { 779*6236dae4SAndroid Build Coastguard Worker # FTP server has IPv6 support! 780*6236dae4SAndroid Build Coastguard Worker $ftp_ipv6 = 1; 781*6236dae4SAndroid Build Coastguard Worker } 782*6236dae4SAndroid Build Coastguard Worker } 783*6236dae4SAndroid Build Coastguard Worker 784*6236dae4SAndroid Build Coastguard Worker if($feature{"UnixSockets"}) { 785*6236dae4SAndroid Build Coastguard Worker # client has Unix sockets support, check whether the HTTP server has it 786*6236dae4SAndroid Build Coastguard Worker my $cmd = "server/sws".exe_ext('SRV')." --version"; 787*6236dae4SAndroid Build Coastguard Worker my @sws = `$cmd`; 788*6236dae4SAndroid Build Coastguard Worker $http_unix = 1 if($sws[0] =~ /unix/); 789*6236dae4SAndroid Build Coastguard Worker } 790*6236dae4SAndroid Build Coastguard Worker 791*6236dae4SAndroid Build Coastguard Worker open(my $manh, "-|", shell_quote($CURL) . " -M 2>&1"); 792*6236dae4SAndroid Build Coastguard Worker while(my $s = <$manh>) { 793*6236dae4SAndroid Build Coastguard Worker if($s =~ /built-in manual was disabled at build-time/) { 794*6236dae4SAndroid Build Coastguard Worker $feature{"manual"} = 0; 795*6236dae4SAndroid Build Coastguard Worker last; 796*6236dae4SAndroid Build Coastguard Worker } 797*6236dae4SAndroid Build Coastguard Worker $feature{"manual"} = 1; 798*6236dae4SAndroid Build Coastguard Worker last; 799*6236dae4SAndroid Build Coastguard Worker } 800*6236dae4SAndroid Build Coastguard Worker close($manh); 801*6236dae4SAndroid Build Coastguard Worker 802*6236dae4SAndroid Build Coastguard Worker $feature{"unittest"} = $feature{"Debug"}; 803*6236dae4SAndroid Build Coastguard Worker $feature{"nghttpx"} = !!$ENV{'NGHTTPX'}; 804*6236dae4SAndroid Build Coastguard Worker $feature{"nghttpx-h3"} = !!$nghttpx_h3; 805*6236dae4SAndroid Build Coastguard Worker 806*6236dae4SAndroid Build Coastguard Worker # 807*6236dae4SAndroid Build Coastguard Worker # strings that must exactly match the names used in server/disabled.c 808*6236dae4SAndroid Build Coastguard Worker # 809*6236dae4SAndroid Build Coastguard Worker $feature{"cookies"} = 1; 810*6236dae4SAndroid Build Coastguard Worker # Use this as a proxy for any cryptographic authentication 811*6236dae4SAndroid Build Coastguard Worker $feature{"crypto"} = $feature{"NTLM"} || $feature{"Kerberos"} || $feature{"SPNEGO"}; 812*6236dae4SAndroid Build Coastguard Worker $feature{"DoH"} = 1; 813*6236dae4SAndroid Build Coastguard Worker $feature{"HTTP-auth"} = 1; 814*6236dae4SAndroid Build Coastguard Worker $feature{"Mime"} = 1; 815*6236dae4SAndroid Build Coastguard Worker $feature{"form-api"} = 1; 816*6236dae4SAndroid Build Coastguard Worker $feature{"netrc"} = 1; 817*6236dae4SAndroid Build Coastguard Worker $feature{"parsedate"} = 1; 818*6236dae4SAndroid Build Coastguard Worker $feature{"proxy"} = 1; 819*6236dae4SAndroid Build Coastguard Worker $feature{"shuffle-dns"} = 1; 820*6236dae4SAndroid Build Coastguard Worker $feature{"typecheck"} = 1; 821*6236dae4SAndroid Build Coastguard Worker $feature{"verbose-strings"} = 1; 822*6236dae4SAndroid Build Coastguard Worker $feature{"wakeup"} = 1; 823*6236dae4SAndroid Build Coastguard Worker $feature{"headers-api"} = 1; 824*6236dae4SAndroid Build Coastguard Worker $feature{"xattr"} = 1; 825*6236dae4SAndroid Build Coastguard Worker $feature{"large-time"} = 1; 826*6236dae4SAndroid Build Coastguard Worker $feature{"sha512-256"} = 1; 827*6236dae4SAndroid Build Coastguard Worker $feature{"local-http"} = servers::localhttp(); 828*6236dae4SAndroid Build Coastguard Worker $feature{"codeset-utf8"} = lc(langinfo(CODESET())) eq "utf-8"; 829*6236dae4SAndroid Build Coastguard Worker 830*6236dae4SAndroid Build Coastguard Worker # make each protocol an enabled "feature" 831*6236dae4SAndroid Build Coastguard Worker for my $p (@protocols) { 832*6236dae4SAndroid Build Coastguard Worker $feature{$p} = 1; 833*6236dae4SAndroid Build Coastguard Worker } 834*6236dae4SAndroid Build Coastguard Worker # 'socks' was once here but is now removed 835*6236dae4SAndroid Build Coastguard Worker 836*6236dae4SAndroid Build Coastguard Worker $has_shared = `sh $CURLCONFIG --built-shared`; 837*6236dae4SAndroid Build Coastguard Worker chomp $has_shared; 838*6236dae4SAndroid Build Coastguard Worker $has_shared = $has_shared eq "yes"; 839*6236dae4SAndroid Build Coastguard Worker 840*6236dae4SAndroid Build Coastguard Worker if(!$feature{"TrackMemory"} && $torture) { 841*6236dae4SAndroid Build Coastguard Worker die "can't run torture tests since curl was built without ". 842*6236dae4SAndroid Build Coastguard Worker "TrackMemory feature (--enable-curldebug)"; 843*6236dae4SAndroid Build Coastguard Worker } 844*6236dae4SAndroid Build Coastguard Worker 845*6236dae4SAndroid Build Coastguard Worker my $hostname=join(' ', runclientoutput("hostname")); 846*6236dae4SAndroid Build Coastguard Worker chomp $hostname; 847*6236dae4SAndroid Build Coastguard Worker my $hosttype=join(' ', runclientoutput("uname -a")); 848*6236dae4SAndroid Build Coastguard Worker chomp $hosttype; 849*6236dae4SAndroid Build Coastguard Worker my $hostos=$^O; 850*6236dae4SAndroid Build Coastguard Worker 851*6236dae4SAndroid Build Coastguard Worker # display summary information about curl and the test host 852*6236dae4SAndroid Build Coastguard Worker logmsg ("********* System characteristics ******** \n", 853*6236dae4SAndroid Build Coastguard Worker "* $curl\n", 854*6236dae4SAndroid Build Coastguard Worker "* $libcurl\n", 855*6236dae4SAndroid Build Coastguard Worker "* Protocols: $proto\n", 856*6236dae4SAndroid Build Coastguard Worker "* Features: $feat\n", 857*6236dae4SAndroid Build Coastguard Worker "* Disabled: $dis\n", 858*6236dae4SAndroid Build Coastguard Worker "* Host: $hostname\n", 859*6236dae4SAndroid Build Coastguard Worker "* System: $hosttype\n", 860*6236dae4SAndroid Build Coastguard Worker "* OS: $hostos\n", 861*6236dae4SAndroid Build Coastguard Worker "* Perl: $^V ($^X)\n", 862*6236dae4SAndroid Build Coastguard Worker "* Args: $args\n"); 863*6236dae4SAndroid Build Coastguard Worker 864*6236dae4SAndroid Build Coastguard Worker if($jobs) { 865*6236dae4SAndroid Build Coastguard Worker # Only show if not the default for now 866*6236dae4SAndroid Build Coastguard Worker logmsg "* Jobs: $jobs\n"; 867*6236dae4SAndroid Build Coastguard Worker } 868*6236dae4SAndroid Build Coastguard Worker if($feature{"TrackMemory"} && $feature{"threaded-resolver"}) { 869*6236dae4SAndroid Build Coastguard Worker logmsg("*\n", 870*6236dae4SAndroid Build Coastguard Worker "*** DISABLES memory tracking when using threaded resolver\n", 871*6236dae4SAndroid Build Coastguard Worker "*\n"); 872*6236dae4SAndroid Build Coastguard Worker } 873*6236dae4SAndroid Build Coastguard Worker 874*6236dae4SAndroid Build Coastguard Worker logmsg sprintf("* Env: %s%s%s%s", $valgrind?"Valgrind ":"", 875*6236dae4SAndroid Build Coastguard Worker $run_event_based?"event-based ":"", 876*6236dae4SAndroid Build Coastguard Worker $bundle?"bundle ":"", 877*6236dae4SAndroid Build Coastguard Worker $nghttpx_h3); 878*6236dae4SAndroid Build Coastguard Worker logmsg sprintf("%s\n", $libtool?"Libtool ":""); 879*6236dae4SAndroid Build Coastguard Worker logmsg ("* Seed: $randseed\n"); 880*6236dae4SAndroid Build Coastguard Worker 881*6236dae4SAndroid Build Coastguard Worker # Disable memory tracking when using threaded resolver 882*6236dae4SAndroid Build Coastguard Worker $feature{"TrackMemory"} = $feature{"TrackMemory"} && !$feature{"threaded-resolver"}; 883*6236dae4SAndroid Build Coastguard Worker 884*6236dae4SAndroid Build Coastguard Worker # toggle off the features that were disabled in the build 885*6236dae4SAndroid Build Coastguard Worker for my $d(@disabled) { 886*6236dae4SAndroid Build Coastguard Worker $feature{$d} = 0; 887*6236dae4SAndroid Build Coastguard Worker } 888*6236dae4SAndroid Build Coastguard Worker} 889*6236dae4SAndroid Build Coastguard Worker 890*6236dae4SAndroid Build Coastguard Worker####################################################################### 891*6236dae4SAndroid Build Coastguard Worker# display information about server features 892*6236dae4SAndroid Build Coastguard Worker# 893*6236dae4SAndroid Build Coastguard Workersub displayserverfeatures { 894*6236dae4SAndroid Build Coastguard Worker logmsg sprintf("* Servers: %s", $stunnel?"SSL ":""); 895*6236dae4SAndroid Build Coastguard Worker logmsg sprintf("%s", $http_ipv6?"HTTP-IPv6 ":""); 896*6236dae4SAndroid Build Coastguard Worker logmsg sprintf("%s", $http_unix?"HTTP-unix ":""); 897*6236dae4SAndroid Build Coastguard Worker logmsg sprintf("%s\n", $ftp_ipv6?"FTP-IPv6 ":""); 898*6236dae4SAndroid Build Coastguard Worker logmsg "***************************************** \n"; 899*6236dae4SAndroid Build Coastguard Worker} 900*6236dae4SAndroid Build Coastguard Worker 901*6236dae4SAndroid Build Coastguard Worker####################################################################### 902*6236dae4SAndroid Build Coastguard Worker# Provide time stamps for single test skipped events 903*6236dae4SAndroid Build Coastguard Worker# 904*6236dae4SAndroid Build Coastguard Workersub timestampskippedevents { 905*6236dae4SAndroid Build Coastguard Worker my $testnum = $_[0]; 906*6236dae4SAndroid Build Coastguard Worker 907*6236dae4SAndroid Build Coastguard Worker return if((not defined($testnum)) || ($testnum < 1)); 908*6236dae4SAndroid Build Coastguard Worker 909*6236dae4SAndroid Build Coastguard Worker if($timestats) { 910*6236dae4SAndroid Build Coastguard Worker 911*6236dae4SAndroid Build Coastguard Worker if($timevrfyend{$testnum}) { 912*6236dae4SAndroid Build Coastguard Worker return; 913*6236dae4SAndroid Build Coastguard Worker } 914*6236dae4SAndroid Build Coastguard Worker elsif($timesrvrlog{$testnum}) { 915*6236dae4SAndroid Build Coastguard Worker $timevrfyend{$testnum} = $timesrvrlog{$testnum}; 916*6236dae4SAndroid Build Coastguard Worker return; 917*6236dae4SAndroid Build Coastguard Worker } 918*6236dae4SAndroid Build Coastguard Worker elsif($timetoolend{$testnum}) { 919*6236dae4SAndroid Build Coastguard Worker $timevrfyend{$testnum} = $timetoolend{$testnum}; 920*6236dae4SAndroid Build Coastguard Worker $timesrvrlog{$testnum} = $timetoolend{$testnum}; 921*6236dae4SAndroid Build Coastguard Worker } 922*6236dae4SAndroid Build Coastguard Worker elsif($timetoolini{$testnum}) { 923*6236dae4SAndroid Build Coastguard Worker $timevrfyend{$testnum} = $timetoolini{$testnum}; 924*6236dae4SAndroid Build Coastguard Worker $timesrvrlog{$testnum} = $timetoolini{$testnum}; 925*6236dae4SAndroid Build Coastguard Worker $timetoolend{$testnum} = $timetoolini{$testnum}; 926*6236dae4SAndroid Build Coastguard Worker } 927*6236dae4SAndroid Build Coastguard Worker elsif($timesrvrend{$testnum}) { 928*6236dae4SAndroid Build Coastguard Worker $timevrfyend{$testnum} = $timesrvrend{$testnum}; 929*6236dae4SAndroid Build Coastguard Worker $timesrvrlog{$testnum} = $timesrvrend{$testnum}; 930*6236dae4SAndroid Build Coastguard Worker $timetoolend{$testnum} = $timesrvrend{$testnum}; 931*6236dae4SAndroid Build Coastguard Worker $timetoolini{$testnum} = $timesrvrend{$testnum}; 932*6236dae4SAndroid Build Coastguard Worker } 933*6236dae4SAndroid Build Coastguard Worker elsif($timesrvrini{$testnum}) { 934*6236dae4SAndroid Build Coastguard Worker $timevrfyend{$testnum} = $timesrvrini{$testnum}; 935*6236dae4SAndroid Build Coastguard Worker $timesrvrlog{$testnum} = $timesrvrini{$testnum}; 936*6236dae4SAndroid Build Coastguard Worker $timetoolend{$testnum} = $timesrvrini{$testnum}; 937*6236dae4SAndroid Build Coastguard Worker $timetoolini{$testnum} = $timesrvrini{$testnum}; 938*6236dae4SAndroid Build Coastguard Worker $timesrvrend{$testnum} = $timesrvrini{$testnum}; 939*6236dae4SAndroid Build Coastguard Worker } 940*6236dae4SAndroid Build Coastguard Worker elsif($timeprepini{$testnum}) { 941*6236dae4SAndroid Build Coastguard Worker $timevrfyend{$testnum} = $timeprepini{$testnum}; 942*6236dae4SAndroid Build Coastguard Worker $timesrvrlog{$testnum} = $timeprepini{$testnum}; 943*6236dae4SAndroid Build Coastguard Worker $timetoolend{$testnum} = $timeprepini{$testnum}; 944*6236dae4SAndroid Build Coastguard Worker $timetoolini{$testnum} = $timeprepini{$testnum}; 945*6236dae4SAndroid Build Coastguard Worker $timesrvrend{$testnum} = $timeprepini{$testnum}; 946*6236dae4SAndroid Build Coastguard Worker $timesrvrini{$testnum} = $timeprepini{$testnum}; 947*6236dae4SAndroid Build Coastguard Worker } 948*6236dae4SAndroid Build Coastguard Worker } 949*6236dae4SAndroid Build Coastguard Worker} 950*6236dae4SAndroid Build Coastguard Worker 951*6236dae4SAndroid Build Coastguard Worker 952*6236dae4SAndroid Build Coastguard Worker# Setup CI Test Run 953*6236dae4SAndroid Build Coastguard Workersub citest_starttestrun { 954*6236dae4SAndroid Build Coastguard Worker if(azure_check_environment()) { 955*6236dae4SAndroid Build Coastguard Worker $AZURE_RUN_ID = azure_create_test_run($ACURL); 956*6236dae4SAndroid Build Coastguard Worker logmsg "Azure Run ID: $AZURE_RUN_ID\n" if ($verbose); 957*6236dae4SAndroid Build Coastguard Worker } 958*6236dae4SAndroid Build Coastguard Worker # Appveyor doesn't require anything here 959*6236dae4SAndroid Build Coastguard Worker} 960*6236dae4SAndroid Build Coastguard Worker 961*6236dae4SAndroid Build Coastguard Worker 962*6236dae4SAndroid Build Coastguard Worker# Register the test case with the CI runner 963*6236dae4SAndroid Build Coastguard Workersub citest_starttest { 964*6236dae4SAndroid Build Coastguard Worker my $testnum = $_[0]; 965*6236dae4SAndroid Build Coastguard Worker 966*6236dae4SAndroid Build Coastguard Worker # get the name of the test early 967*6236dae4SAndroid Build Coastguard Worker my $testname= (getpart("client", "name"))[0]; 968*6236dae4SAndroid Build Coastguard Worker chomp $testname; 969*6236dae4SAndroid Build Coastguard Worker 970*6236dae4SAndroid Build Coastguard Worker # create test result in CI services 971*6236dae4SAndroid Build Coastguard Worker if(azure_check_environment() && $AZURE_RUN_ID) { 972*6236dae4SAndroid Build Coastguard Worker $AZURE_RESULT_ID = azure_create_test_result($ACURL, $AZURE_RUN_ID, $testnum, $testname); 973*6236dae4SAndroid Build Coastguard Worker } 974*6236dae4SAndroid Build Coastguard Worker elsif(appveyor_check_environment()) { 975*6236dae4SAndroid Build Coastguard Worker appveyor_create_test_result($ACURL, $testnum, $testname); 976*6236dae4SAndroid Build Coastguard Worker } 977*6236dae4SAndroid Build Coastguard Worker} 978*6236dae4SAndroid Build Coastguard Worker 979*6236dae4SAndroid Build Coastguard Worker 980*6236dae4SAndroid Build Coastguard Worker# Submit the test case result with the CI runner 981*6236dae4SAndroid Build Coastguard Workersub citest_finishtest { 982*6236dae4SAndroid Build Coastguard Worker my ($testnum, $error) = @_; 983*6236dae4SAndroid Build Coastguard Worker # update test result in CI services 984*6236dae4SAndroid Build Coastguard Worker if(azure_check_environment() && $AZURE_RUN_ID && $AZURE_RESULT_ID) { 985*6236dae4SAndroid Build Coastguard Worker $AZURE_RESULT_ID = azure_update_test_result($ACURL, $AZURE_RUN_ID, $AZURE_RESULT_ID, $testnum, $error, 986*6236dae4SAndroid Build Coastguard Worker $timeprepini{$testnum}, $timevrfyend{$testnum}); 987*6236dae4SAndroid Build Coastguard Worker } 988*6236dae4SAndroid Build Coastguard Worker elsif(appveyor_check_environment()) { 989*6236dae4SAndroid Build Coastguard Worker appveyor_update_test_result($ACURL, $testnum, $error, $timeprepini{$testnum}, $timevrfyend{$testnum}); 990*6236dae4SAndroid Build Coastguard Worker } 991*6236dae4SAndroid Build Coastguard Worker} 992*6236dae4SAndroid Build Coastguard Worker 993*6236dae4SAndroid Build Coastguard Worker# Complete CI test run 994*6236dae4SAndroid Build Coastguard Workersub citest_finishtestrun { 995*6236dae4SAndroid Build Coastguard Worker if(azure_check_environment() && $AZURE_RUN_ID) { 996*6236dae4SAndroid Build Coastguard Worker $AZURE_RUN_ID = azure_update_test_run($ACURL, $AZURE_RUN_ID); 997*6236dae4SAndroid Build Coastguard Worker } 998*6236dae4SAndroid Build Coastguard Worker # Appveyor doesn't require anything here 999*6236dae4SAndroid Build Coastguard Worker} 1000*6236dae4SAndroid Build Coastguard Worker 1001*6236dae4SAndroid Build Coastguard Worker 1002*6236dae4SAndroid Build Coastguard Worker# add one set of test timings from the runner to global set 1003*6236dae4SAndroid Build Coastguard Workersub updatetesttimings { 1004*6236dae4SAndroid Build Coastguard Worker my ($testnum, %testtimings)=@_; 1005*6236dae4SAndroid Build Coastguard Worker 1006*6236dae4SAndroid Build Coastguard Worker if(defined $testtimings{"timeprepini"}) { 1007*6236dae4SAndroid Build Coastguard Worker $timeprepini{$testnum} = $testtimings{"timeprepini"}; 1008*6236dae4SAndroid Build Coastguard Worker } 1009*6236dae4SAndroid Build Coastguard Worker if(defined $testtimings{"timesrvrini"}) { 1010*6236dae4SAndroid Build Coastguard Worker $timesrvrini{$testnum} = $testtimings{"timesrvrini"}; 1011*6236dae4SAndroid Build Coastguard Worker } 1012*6236dae4SAndroid Build Coastguard Worker if(defined $testtimings{"timesrvrend"}) { 1013*6236dae4SAndroid Build Coastguard Worker $timesrvrend{$testnum} = $testtimings{"timesrvrend"}; 1014*6236dae4SAndroid Build Coastguard Worker } 1015*6236dae4SAndroid Build Coastguard Worker if(defined $testtimings{"timetoolini"}) { 1016*6236dae4SAndroid Build Coastguard Worker $timetoolini{$testnum} = $testtimings{"timetoolini"}; 1017*6236dae4SAndroid Build Coastguard Worker } 1018*6236dae4SAndroid Build Coastguard Worker if(defined $testtimings{"timetoolend"}) { 1019*6236dae4SAndroid Build Coastguard Worker $timetoolend{$testnum} = $testtimings{"timetoolend"}; 1020*6236dae4SAndroid Build Coastguard Worker } 1021*6236dae4SAndroid Build Coastguard Worker if(defined $testtimings{"timesrvrlog"}) { 1022*6236dae4SAndroid Build Coastguard Worker $timesrvrlog{$testnum} = $testtimings{"timesrvrlog"}; 1023*6236dae4SAndroid Build Coastguard Worker } 1024*6236dae4SAndroid Build Coastguard Worker} 1025*6236dae4SAndroid Build Coastguard Worker 1026*6236dae4SAndroid Build Coastguard Worker 1027*6236dae4SAndroid Build Coastguard Worker####################################################################### 1028*6236dae4SAndroid Build Coastguard Worker# Return the log directory for the given test runner 1029*6236dae4SAndroid Build Coastguard Workersub getrunnernumlogdir { 1030*6236dae4SAndroid Build Coastguard Worker my $runnernum = $_[0]; 1031*6236dae4SAndroid Build Coastguard Worker return $jobs > 1 ? "$LOGDIR/$runnernum" : $LOGDIR; 1032*6236dae4SAndroid Build Coastguard Worker} 1033*6236dae4SAndroid Build Coastguard Worker 1034*6236dae4SAndroid Build Coastguard Worker####################################################################### 1035*6236dae4SAndroid Build Coastguard Worker# Return the log directory for the given test runner ID 1036*6236dae4SAndroid Build Coastguard Workersub getrunnerlogdir { 1037*6236dae4SAndroid Build Coastguard Worker my $runnerid = $_[0]; 1038*6236dae4SAndroid Build Coastguard Worker if($jobs <= 1) { 1039*6236dae4SAndroid Build Coastguard Worker return $LOGDIR; 1040*6236dae4SAndroid Build Coastguard Worker } 1041*6236dae4SAndroid Build Coastguard Worker # TODO: speed up this O(n) operation 1042*6236dae4SAndroid Build Coastguard Worker for my $runnernum (keys %runnerids) { 1043*6236dae4SAndroid Build Coastguard Worker if($runnerid eq $runnerids{$runnernum}) { 1044*6236dae4SAndroid Build Coastguard Worker return "$LOGDIR/$runnernum"; 1045*6236dae4SAndroid Build Coastguard Worker } 1046*6236dae4SAndroid Build Coastguard Worker } 1047*6236dae4SAndroid Build Coastguard Worker die "Internal error: runner ID $runnerid not found"; 1048*6236dae4SAndroid Build Coastguard Worker} 1049*6236dae4SAndroid Build Coastguard Worker 1050*6236dae4SAndroid Build Coastguard Worker 1051*6236dae4SAndroid Build Coastguard Worker####################################################################### 1052*6236dae4SAndroid Build Coastguard Worker# Verify that this test case should be run 1053*6236dae4SAndroid Build Coastguard Workersub singletest_shouldrun { 1054*6236dae4SAndroid Build Coastguard Worker my $testnum = $_[0]; 1055*6236dae4SAndroid Build Coastguard Worker my $why; # why the test won't be run 1056*6236dae4SAndroid Build Coastguard Worker my $errorreturncode = 1; # 1 means normal error, 2 means ignored error 1057*6236dae4SAndroid Build Coastguard Worker my @what; # what features are needed 1058*6236dae4SAndroid Build Coastguard Worker 1059*6236dae4SAndroid Build Coastguard Worker if($disttests !~ /test$testnum(\W|\z)/ ) { 1060*6236dae4SAndroid Build Coastguard Worker logmsg "Warning: test$testnum not present in tests/data/Makefile.am\n"; 1061*6236dae4SAndroid Build Coastguard Worker } 1062*6236dae4SAndroid Build Coastguard Worker if($disabled{$testnum}) { 1063*6236dae4SAndroid Build Coastguard Worker if(!$run_disabled) { 1064*6236dae4SAndroid Build Coastguard Worker $why = "listed in DISABLED"; 1065*6236dae4SAndroid Build Coastguard Worker } 1066*6236dae4SAndroid Build Coastguard Worker else { 1067*6236dae4SAndroid Build Coastguard Worker logmsg "Warning: test$testnum is explicitly disabled\n"; 1068*6236dae4SAndroid Build Coastguard Worker } 1069*6236dae4SAndroid Build Coastguard Worker } 1070*6236dae4SAndroid Build Coastguard Worker if($ignored{$testnum}) { 1071*6236dae4SAndroid Build Coastguard Worker logmsg "Warning: test$testnum result is ignored\n"; 1072*6236dae4SAndroid Build Coastguard Worker $errorreturncode = 2; 1073*6236dae4SAndroid Build Coastguard Worker } 1074*6236dae4SAndroid Build Coastguard Worker 1075*6236dae4SAndroid Build Coastguard Worker if(loadtest("${TESTDIR}/test${testnum}")) { 1076*6236dae4SAndroid Build Coastguard Worker if($verbose) { 1077*6236dae4SAndroid Build Coastguard Worker # this is not a test 1078*6236dae4SAndroid Build Coastguard Worker logmsg "RUN: $testnum doesn't look like a test case\n"; 1079*6236dae4SAndroid Build Coastguard Worker } 1080*6236dae4SAndroid Build Coastguard Worker $why = "no test"; 1081*6236dae4SAndroid Build Coastguard Worker } 1082*6236dae4SAndroid Build Coastguard Worker else { 1083*6236dae4SAndroid Build Coastguard Worker @what = getpart("client", "features"); 1084*6236dae4SAndroid Build Coastguard Worker } 1085*6236dae4SAndroid Build Coastguard Worker 1086*6236dae4SAndroid Build Coastguard Worker # We require a feature to be present 1087*6236dae4SAndroid Build Coastguard Worker for(@what) { 1088*6236dae4SAndroid Build Coastguard Worker my $f = $_; 1089*6236dae4SAndroid Build Coastguard Worker $f =~ s/\s//g; 1090*6236dae4SAndroid Build Coastguard Worker 1091*6236dae4SAndroid Build Coastguard Worker if($f =~ /^([^!].*)$/) { 1092*6236dae4SAndroid Build Coastguard Worker if($feature{$1}) { 1093*6236dae4SAndroid Build Coastguard Worker next; 1094*6236dae4SAndroid Build Coastguard Worker } 1095*6236dae4SAndroid Build Coastguard Worker 1096*6236dae4SAndroid Build Coastguard Worker $why = "curl lacks $1 support"; 1097*6236dae4SAndroid Build Coastguard Worker last; 1098*6236dae4SAndroid Build Coastguard Worker } 1099*6236dae4SAndroid Build Coastguard Worker } 1100*6236dae4SAndroid Build Coastguard Worker 1101*6236dae4SAndroid Build Coastguard Worker # We require a feature to not be present 1102*6236dae4SAndroid Build Coastguard Worker if(!$why) { 1103*6236dae4SAndroid Build Coastguard Worker for(@what) { 1104*6236dae4SAndroid Build Coastguard Worker my $f = $_; 1105*6236dae4SAndroid Build Coastguard Worker $f =~ s/\s//g; 1106*6236dae4SAndroid Build Coastguard Worker 1107*6236dae4SAndroid Build Coastguard Worker if($f =~ /^!(.*)$/) { 1108*6236dae4SAndroid Build Coastguard Worker if(!$feature{$1}) { 1109*6236dae4SAndroid Build Coastguard Worker next; 1110*6236dae4SAndroid Build Coastguard Worker } 1111*6236dae4SAndroid Build Coastguard Worker } 1112*6236dae4SAndroid Build Coastguard Worker else { 1113*6236dae4SAndroid Build Coastguard Worker next; 1114*6236dae4SAndroid Build Coastguard Worker } 1115*6236dae4SAndroid Build Coastguard Worker 1116*6236dae4SAndroid Build Coastguard Worker $why = "curl has $1 support"; 1117*6236dae4SAndroid Build Coastguard Worker last; 1118*6236dae4SAndroid Build Coastguard Worker } 1119*6236dae4SAndroid Build Coastguard Worker } 1120*6236dae4SAndroid Build Coastguard Worker 1121*6236dae4SAndroid Build Coastguard Worker my @info_keywords; 1122*6236dae4SAndroid Build Coastguard Worker if(!$why) { 1123*6236dae4SAndroid Build Coastguard Worker @info_keywords = getpart("info", "keywords"); 1124*6236dae4SAndroid Build Coastguard Worker 1125*6236dae4SAndroid Build Coastguard Worker if(!$info_keywords[0]) { 1126*6236dae4SAndroid Build Coastguard Worker $why = "missing the <keywords> section!"; 1127*6236dae4SAndroid Build Coastguard Worker } 1128*6236dae4SAndroid Build Coastguard Worker 1129*6236dae4SAndroid Build Coastguard Worker my $match; 1130*6236dae4SAndroid Build Coastguard Worker for my $k (@info_keywords) { 1131*6236dae4SAndroid Build Coastguard Worker chomp $k; 1132*6236dae4SAndroid Build Coastguard Worker if ($disabled_keywords{lc($k)}) { 1133*6236dae4SAndroid Build Coastguard Worker $why = "disabled by keyword"; 1134*6236dae4SAndroid Build Coastguard Worker } 1135*6236dae4SAndroid Build Coastguard Worker elsif ($enabled_keywords{lc($k)}) { 1136*6236dae4SAndroid Build Coastguard Worker $match = 1; 1137*6236dae4SAndroid Build Coastguard Worker } 1138*6236dae4SAndroid Build Coastguard Worker if ($ignored_keywords{lc($k)}) { 1139*6236dae4SAndroid Build Coastguard Worker logmsg "Warning: test$testnum result is ignored due to $k\n"; 1140*6236dae4SAndroid Build Coastguard Worker $errorreturncode = 2; 1141*6236dae4SAndroid Build Coastguard Worker } 1142*6236dae4SAndroid Build Coastguard Worker } 1143*6236dae4SAndroid Build Coastguard Worker 1144*6236dae4SAndroid Build Coastguard Worker if(!$why && !$match && %enabled_keywords) { 1145*6236dae4SAndroid Build Coastguard Worker $why = "disabled by missing keyword"; 1146*6236dae4SAndroid Build Coastguard Worker } 1147*6236dae4SAndroid Build Coastguard Worker } 1148*6236dae4SAndroid Build Coastguard Worker 1149*6236dae4SAndroid Build Coastguard Worker if (!$why && defined $custom_skip_reasons{test}{$testnum}) { 1150*6236dae4SAndroid Build Coastguard Worker $why = $custom_skip_reasons{test}{$testnum}; 1151*6236dae4SAndroid Build Coastguard Worker } 1152*6236dae4SAndroid Build Coastguard Worker 1153*6236dae4SAndroid Build Coastguard Worker if (!$why && defined $custom_skip_reasons{tool}) { 1154*6236dae4SAndroid Build Coastguard Worker foreach my $tool (getpart("client", "tool")) { 1155*6236dae4SAndroid Build Coastguard Worker foreach my $tool_skip_pattern (keys %{$custom_skip_reasons{tool}}) { 1156*6236dae4SAndroid Build Coastguard Worker if ($tool =~ /$tool_skip_pattern/i) { 1157*6236dae4SAndroid Build Coastguard Worker $why = $custom_skip_reasons{tool}{$tool_skip_pattern}; 1158*6236dae4SAndroid Build Coastguard Worker } 1159*6236dae4SAndroid Build Coastguard Worker } 1160*6236dae4SAndroid Build Coastguard Worker } 1161*6236dae4SAndroid Build Coastguard Worker } 1162*6236dae4SAndroid Build Coastguard Worker 1163*6236dae4SAndroid Build Coastguard Worker if (!$why && defined $custom_skip_reasons{keyword}) { 1164*6236dae4SAndroid Build Coastguard Worker foreach my $keyword (@info_keywords) { 1165*6236dae4SAndroid Build Coastguard Worker foreach my $keyword_skip_pattern (keys %{$custom_skip_reasons{keyword}}) { 1166*6236dae4SAndroid Build Coastguard Worker if ($keyword =~ /$keyword_skip_pattern/i) { 1167*6236dae4SAndroid Build Coastguard Worker $why = $custom_skip_reasons{keyword}{$keyword_skip_pattern}; 1168*6236dae4SAndroid Build Coastguard Worker } 1169*6236dae4SAndroid Build Coastguard Worker } 1170*6236dae4SAndroid Build Coastguard Worker } 1171*6236dae4SAndroid Build Coastguard Worker } 1172*6236dae4SAndroid Build Coastguard Worker 1173*6236dae4SAndroid Build Coastguard Worker return ($why, $errorreturncode); 1174*6236dae4SAndroid Build Coastguard Worker} 1175*6236dae4SAndroid Build Coastguard Worker 1176*6236dae4SAndroid Build Coastguard Worker 1177*6236dae4SAndroid Build Coastguard Worker####################################################################### 1178*6236dae4SAndroid Build Coastguard Worker# Print the test name and count tests 1179*6236dae4SAndroid Build Coastguard Workersub singletest_count { 1180*6236dae4SAndroid Build Coastguard Worker my ($testnum, $why) = @_; 1181*6236dae4SAndroid Build Coastguard Worker 1182*6236dae4SAndroid Build Coastguard Worker if($why && !$listonly) { 1183*6236dae4SAndroid Build Coastguard Worker # there's a problem, count it as "skipped" 1184*6236dae4SAndroid Build Coastguard Worker $skipped{$why}++; 1185*6236dae4SAndroid Build Coastguard Worker $teststat[$testnum]=$why; # store reason for this test case 1186*6236dae4SAndroid Build Coastguard Worker 1187*6236dae4SAndroid Build Coastguard Worker if(!$short) { 1188*6236dae4SAndroid Build Coastguard Worker if($skipped{$why} <= 3) { 1189*6236dae4SAndroid Build Coastguard Worker # show only the first three skips for each reason 1190*6236dae4SAndroid Build Coastguard Worker logmsg sprintf("test %04d SKIPPED: $why\n", $testnum); 1191*6236dae4SAndroid Build Coastguard Worker } 1192*6236dae4SAndroid Build Coastguard Worker } 1193*6236dae4SAndroid Build Coastguard Worker 1194*6236dae4SAndroid Build Coastguard Worker timestampskippedevents($testnum); 1195*6236dae4SAndroid Build Coastguard Worker return -1; 1196*6236dae4SAndroid Build Coastguard Worker } 1197*6236dae4SAndroid Build Coastguard Worker 1198*6236dae4SAndroid Build Coastguard Worker # At this point we've committed to run this test 1199*6236dae4SAndroid Build Coastguard Worker logmsg sprintf("test %04d...", $testnum) if(!$automakestyle); 1200*6236dae4SAndroid Build Coastguard Worker 1201*6236dae4SAndroid Build Coastguard Worker # name of the test 1202*6236dae4SAndroid Build Coastguard Worker my $testname= (getpart("client", "name"))[0]; 1203*6236dae4SAndroid Build Coastguard Worker chomp $testname; 1204*6236dae4SAndroid Build Coastguard Worker logmsg "[$testname]\n" if(!$short); 1205*6236dae4SAndroid Build Coastguard Worker 1206*6236dae4SAndroid Build Coastguard Worker if($listonly) { 1207*6236dae4SAndroid Build Coastguard Worker timestampskippedevents($testnum); 1208*6236dae4SAndroid Build Coastguard Worker } 1209*6236dae4SAndroid Build Coastguard Worker return 0; 1210*6236dae4SAndroid Build Coastguard Worker} 1211*6236dae4SAndroid Build Coastguard Worker 1212*6236dae4SAndroid Build Coastguard Worker# Make sure all line endings in the array are the same: CRLF 1213*6236dae4SAndroid Build Coastguard Workersub normalize_text { 1214*6236dae4SAndroid Build Coastguard Worker my ($ref) = @_; 1215*6236dae4SAndroid Build Coastguard Worker s/\r\n/\n/g for @$ref; 1216*6236dae4SAndroid Build Coastguard Worker s/\n/\r\n/g for @$ref; 1217*6236dae4SAndroid Build Coastguard Worker} 1218*6236dae4SAndroid Build Coastguard Worker 1219*6236dae4SAndroid Build Coastguard Worker####################################################################### 1220*6236dae4SAndroid Build Coastguard Worker# Verify test succeeded 1221*6236dae4SAndroid Build Coastguard Workersub singletest_check { 1222*6236dae4SAndroid Build Coastguard Worker my ($runnerid, $testnum, $cmdres, $CURLOUT, $tool, $usedvalgrind)=@_; 1223*6236dae4SAndroid Build Coastguard Worker 1224*6236dae4SAndroid Build Coastguard Worker # Skip all the verification on torture tests 1225*6236dae4SAndroid Build Coastguard Worker if ($torture) { 1226*6236dae4SAndroid Build Coastguard Worker # timestamp test result verification end 1227*6236dae4SAndroid Build Coastguard Worker $timevrfyend{$testnum} = Time::HiRes::time(); 1228*6236dae4SAndroid Build Coastguard Worker return -2; 1229*6236dae4SAndroid Build Coastguard Worker } 1230*6236dae4SAndroid Build Coastguard Worker 1231*6236dae4SAndroid Build Coastguard Worker my $logdir = getrunnerlogdir($runnerid); 1232*6236dae4SAndroid Build Coastguard Worker my @err = getpart("verify", "errorcode"); 1233*6236dae4SAndroid Build Coastguard Worker my $errorcode = $err[0] || "0"; 1234*6236dae4SAndroid Build Coastguard Worker my $ok=""; 1235*6236dae4SAndroid Build Coastguard Worker my $res; 1236*6236dae4SAndroid Build Coastguard Worker chomp $errorcode; 1237*6236dae4SAndroid Build Coastguard Worker my $testname= (getpart("client", "name"))[0]; 1238*6236dae4SAndroid Build Coastguard Worker chomp $testname; 1239*6236dae4SAndroid Build Coastguard Worker # what parts to cut off from stdout/stderr 1240*6236dae4SAndroid Build Coastguard Worker my @stripfile = getpart("verify", "stripfile"); 1241*6236dae4SAndroid Build Coastguard Worker 1242*6236dae4SAndroid Build Coastguard Worker my @validstdout = getpart("verify", "stdout"); 1243*6236dae4SAndroid Build Coastguard Worker # get all attributes 1244*6236dae4SAndroid Build Coastguard Worker my %hash = getpartattr("verify", "stdout"); 1245*6236dae4SAndroid Build Coastguard Worker 1246*6236dae4SAndroid Build Coastguard Worker my $loadfile = $hash{'loadfile'}; 1247*6236dae4SAndroid Build Coastguard Worker if ($loadfile) { 1248*6236dae4SAndroid Build Coastguard Worker open(my $tmp, "<", "$loadfile") || die "Cannot open file $loadfile: $!"; 1249*6236dae4SAndroid Build Coastguard Worker @validstdout = <$tmp>; 1250*6236dae4SAndroid Build Coastguard Worker close($tmp); 1251*6236dae4SAndroid Build Coastguard Worker 1252*6236dae4SAndroid Build Coastguard Worker # Enforce LF newlines on load 1253*6236dae4SAndroid Build Coastguard Worker s/\r\n/\n/g for @validstdout; 1254*6236dae4SAndroid Build Coastguard Worker } 1255*6236dae4SAndroid Build Coastguard Worker 1256*6236dae4SAndroid Build Coastguard Worker if (@validstdout) { 1257*6236dae4SAndroid Build Coastguard Worker # verify redirected stdout 1258*6236dae4SAndroid Build Coastguard Worker my @actual = loadarray(stdoutfilename($logdir, $testnum)); 1259*6236dae4SAndroid Build Coastguard Worker 1260*6236dae4SAndroid Build Coastguard Worker foreach my $strip (@stripfile) { 1261*6236dae4SAndroid Build Coastguard Worker chomp $strip; 1262*6236dae4SAndroid Build Coastguard Worker my @newgen; 1263*6236dae4SAndroid Build Coastguard Worker for(@actual) { 1264*6236dae4SAndroid Build Coastguard Worker eval $strip; 1265*6236dae4SAndroid Build Coastguard Worker if($_) { 1266*6236dae4SAndroid Build Coastguard Worker push @newgen, $_; 1267*6236dae4SAndroid Build Coastguard Worker } 1268*6236dae4SAndroid Build Coastguard Worker } 1269*6236dae4SAndroid Build Coastguard Worker # this is to get rid of array entries that vanished (zero 1270*6236dae4SAndroid Build Coastguard Worker # length) because of replacements 1271*6236dae4SAndroid Build Coastguard Worker @actual = @newgen; 1272*6236dae4SAndroid Build Coastguard Worker } 1273*6236dae4SAndroid Build Coastguard Worker 1274*6236dae4SAndroid Build Coastguard Worker # get the mode attribute 1275*6236dae4SAndroid Build Coastguard Worker my $filemode=$hash{'mode'}; 1276*6236dae4SAndroid Build Coastguard Worker if($filemode && ($filemode eq "text")) { 1277*6236dae4SAndroid Build Coastguard Worker normalize_text(\@validstdout); 1278*6236dae4SAndroid Build Coastguard Worker normalize_text(\@actual); 1279*6236dae4SAndroid Build Coastguard Worker } 1280*6236dae4SAndroid Build Coastguard Worker 1281*6236dae4SAndroid Build Coastguard Worker if($hash{'nonewline'}) { 1282*6236dae4SAndroid Build Coastguard Worker # Yes, we must cut off the final newline from the final line 1283*6236dae4SAndroid Build Coastguard Worker # of the protocol data 1284*6236dae4SAndroid Build Coastguard Worker chomp($validstdout[-1]); 1285*6236dae4SAndroid Build Coastguard Worker } 1286*6236dae4SAndroid Build Coastguard Worker 1287*6236dae4SAndroid Build Coastguard Worker if($hash{'crlf'} || 1288*6236dae4SAndroid Build Coastguard Worker ($feature{"hyper"} && ($keywords{"HTTP"} 1289*6236dae4SAndroid Build Coastguard Worker || $keywords{"HTTPS"}))) { 1290*6236dae4SAndroid Build Coastguard Worker subnewlines(0, \$_) for @validstdout; 1291*6236dae4SAndroid Build Coastguard Worker } 1292*6236dae4SAndroid Build Coastguard Worker 1293*6236dae4SAndroid Build Coastguard Worker $res = compare($runnerid, $testnum, $testname, "stdout", \@actual, \@validstdout); 1294*6236dae4SAndroid Build Coastguard Worker if($res) { 1295*6236dae4SAndroid Build Coastguard Worker return -1; 1296*6236dae4SAndroid Build Coastguard Worker } 1297*6236dae4SAndroid Build Coastguard Worker $ok .= "s"; 1298*6236dae4SAndroid Build Coastguard Worker } 1299*6236dae4SAndroid Build Coastguard Worker else { 1300*6236dae4SAndroid Build Coastguard Worker $ok .= "-"; # stdout not checked 1301*6236dae4SAndroid Build Coastguard Worker } 1302*6236dae4SAndroid Build Coastguard Worker 1303*6236dae4SAndroid Build Coastguard Worker my @validstderr = getpart("verify", "stderr"); 1304*6236dae4SAndroid Build Coastguard Worker if (@validstderr) { 1305*6236dae4SAndroid Build Coastguard Worker # verify redirected stderr 1306*6236dae4SAndroid Build Coastguard Worker my @actual = loadarray(stderrfilename($logdir, $testnum)); 1307*6236dae4SAndroid Build Coastguard Worker 1308*6236dae4SAndroid Build Coastguard Worker foreach my $strip (@stripfile) { 1309*6236dae4SAndroid Build Coastguard Worker chomp $strip; 1310*6236dae4SAndroid Build Coastguard Worker my @newgen; 1311*6236dae4SAndroid Build Coastguard Worker for(@actual) { 1312*6236dae4SAndroid Build Coastguard Worker eval $strip; 1313*6236dae4SAndroid Build Coastguard Worker if($_) { 1314*6236dae4SAndroid Build Coastguard Worker push @newgen, $_; 1315*6236dae4SAndroid Build Coastguard Worker } 1316*6236dae4SAndroid Build Coastguard Worker } 1317*6236dae4SAndroid Build Coastguard Worker # this is to get rid of array entries that vanished (zero 1318*6236dae4SAndroid Build Coastguard Worker # length) because of replacements 1319*6236dae4SAndroid Build Coastguard Worker @actual = @newgen; 1320*6236dae4SAndroid Build Coastguard Worker } 1321*6236dae4SAndroid Build Coastguard Worker 1322*6236dae4SAndroid Build Coastguard Worker # get all attributes 1323*6236dae4SAndroid Build Coastguard Worker my %hash = getpartattr("verify", "stderr"); 1324*6236dae4SAndroid Build Coastguard Worker 1325*6236dae4SAndroid Build Coastguard Worker # get the mode attribute 1326*6236dae4SAndroid Build Coastguard Worker my $filemode=$hash{'mode'}; 1327*6236dae4SAndroid Build Coastguard Worker if($filemode && ($filemode eq "text") && $feature{"hyper"}) { 1328*6236dae4SAndroid Build Coastguard Worker # text mode check in hyper-mode. Sometimes necessary if the stderr 1329*6236dae4SAndroid Build Coastguard Worker # data *looks* like HTTP and thus has gotten CRLF newlines 1330*6236dae4SAndroid Build Coastguard Worker # mistakenly 1331*6236dae4SAndroid Build Coastguard Worker normalize_text(\@validstderr); 1332*6236dae4SAndroid Build Coastguard Worker } 1333*6236dae4SAndroid Build Coastguard Worker if($filemode && ($filemode eq "text")) { 1334*6236dae4SAndroid Build Coastguard Worker normalize_text(\@validstderr); 1335*6236dae4SAndroid Build Coastguard Worker normalize_text(\@actual); 1336*6236dae4SAndroid Build Coastguard Worker } 1337*6236dae4SAndroid Build Coastguard Worker 1338*6236dae4SAndroid Build Coastguard Worker if($hash{'nonewline'}) { 1339*6236dae4SAndroid Build Coastguard Worker # Yes, we must cut off the final newline from the final line 1340*6236dae4SAndroid Build Coastguard Worker # of the protocol data 1341*6236dae4SAndroid Build Coastguard Worker chomp($validstderr[-1]); 1342*6236dae4SAndroid Build Coastguard Worker } 1343*6236dae4SAndroid Build Coastguard Worker 1344*6236dae4SAndroid Build Coastguard Worker if($hash{'crlf'}) { 1345*6236dae4SAndroid Build Coastguard Worker subnewlines(0, \$_) for @validstderr; 1346*6236dae4SAndroid Build Coastguard Worker } 1347*6236dae4SAndroid Build Coastguard Worker 1348*6236dae4SAndroid Build Coastguard Worker $res = compare($runnerid, $testnum, $testname, "stderr", \@actual, \@validstderr); 1349*6236dae4SAndroid Build Coastguard Worker if($res) { 1350*6236dae4SAndroid Build Coastguard Worker return -1; 1351*6236dae4SAndroid Build Coastguard Worker } 1352*6236dae4SAndroid Build Coastguard Worker $ok .= "r"; 1353*6236dae4SAndroid Build Coastguard Worker } 1354*6236dae4SAndroid Build Coastguard Worker else { 1355*6236dae4SAndroid Build Coastguard Worker $ok .= "-"; # stderr not checked 1356*6236dae4SAndroid Build Coastguard Worker } 1357*6236dae4SAndroid Build Coastguard Worker 1358*6236dae4SAndroid Build Coastguard Worker # what to cut off from the live protocol sent by curl 1359*6236dae4SAndroid Build Coastguard Worker my @strip = getpart("verify", "strip"); 1360*6236dae4SAndroid Build Coastguard Worker 1361*6236dae4SAndroid Build Coastguard Worker # what parts to cut off from the protocol & upload 1362*6236dae4SAndroid Build Coastguard Worker my @strippart = getpart("verify", "strippart"); 1363*6236dae4SAndroid Build Coastguard Worker 1364*6236dae4SAndroid Build Coastguard Worker # this is the valid protocol blurb curl should generate 1365*6236dae4SAndroid Build Coastguard Worker my @protocol= getpart("verify", "protocol"); 1366*6236dae4SAndroid Build Coastguard Worker if(@protocol) { 1367*6236dae4SAndroid Build Coastguard Worker # Verify the sent request 1368*6236dae4SAndroid Build Coastguard Worker my @out = loadarray("$logdir/$SERVERIN"); 1369*6236dae4SAndroid Build Coastguard Worker 1370*6236dae4SAndroid Build Coastguard Worker # check if there's any attributes on the verify/protocol section 1371*6236dae4SAndroid Build Coastguard Worker my %hash = getpartattr("verify", "protocol"); 1372*6236dae4SAndroid Build Coastguard Worker 1373*6236dae4SAndroid Build Coastguard Worker if($hash{'nonewline'}) { 1374*6236dae4SAndroid Build Coastguard Worker # Yes, we must cut off the final newline from the final line 1375*6236dae4SAndroid Build Coastguard Worker # of the protocol data 1376*6236dae4SAndroid Build Coastguard Worker chomp($protocol[-1]); 1377*6236dae4SAndroid Build Coastguard Worker } 1378*6236dae4SAndroid Build Coastguard Worker 1379*6236dae4SAndroid Build Coastguard Worker for(@strip) { 1380*6236dae4SAndroid Build Coastguard Worker # strip off all lines that match the patterns from both arrays 1381*6236dae4SAndroid Build Coastguard Worker chomp $_; 1382*6236dae4SAndroid Build Coastguard Worker @out = striparray( $_, \@out); 1383*6236dae4SAndroid Build Coastguard Worker @protocol= striparray( $_, \@protocol); 1384*6236dae4SAndroid Build Coastguard Worker } 1385*6236dae4SAndroid Build Coastguard Worker 1386*6236dae4SAndroid Build Coastguard Worker for my $strip (@strippart) { 1387*6236dae4SAndroid Build Coastguard Worker chomp $strip; 1388*6236dae4SAndroid Build Coastguard Worker for(@out) { 1389*6236dae4SAndroid Build Coastguard Worker eval $strip; 1390*6236dae4SAndroid Build Coastguard Worker } 1391*6236dae4SAndroid Build Coastguard Worker } 1392*6236dae4SAndroid Build Coastguard Worker 1393*6236dae4SAndroid Build Coastguard Worker if($hash{'crlf'}) { 1394*6236dae4SAndroid Build Coastguard Worker subnewlines(1, \$_) for @protocol; 1395*6236dae4SAndroid Build Coastguard Worker } 1396*6236dae4SAndroid Build Coastguard Worker 1397*6236dae4SAndroid Build Coastguard Worker if((!$out[0] || ($out[0] eq "")) && $protocol[0]) { 1398*6236dae4SAndroid Build Coastguard Worker logmsg "\n $testnum: protocol FAILED!\n". 1399*6236dae4SAndroid Build Coastguard Worker " There was no content at all in the file $logdir/$SERVERIN.\n". 1400*6236dae4SAndroid Build Coastguard Worker " Server glitch? Total curl failure? Returned: $cmdres\n"; 1401*6236dae4SAndroid Build Coastguard Worker # timestamp test result verification end 1402*6236dae4SAndroid Build Coastguard Worker $timevrfyend{$testnum} = Time::HiRes::time(); 1403*6236dae4SAndroid Build Coastguard Worker return -1; 1404*6236dae4SAndroid Build Coastguard Worker } 1405*6236dae4SAndroid Build Coastguard Worker 1406*6236dae4SAndroid Build Coastguard Worker $res = compare($runnerid, $testnum, $testname, "protocol", \@out, \@protocol); 1407*6236dae4SAndroid Build Coastguard Worker if($res) { 1408*6236dae4SAndroid Build Coastguard Worker return -1; 1409*6236dae4SAndroid Build Coastguard Worker } 1410*6236dae4SAndroid Build Coastguard Worker 1411*6236dae4SAndroid Build Coastguard Worker $ok .= "p"; 1412*6236dae4SAndroid Build Coastguard Worker 1413*6236dae4SAndroid Build Coastguard Worker } 1414*6236dae4SAndroid Build Coastguard Worker else { 1415*6236dae4SAndroid Build Coastguard Worker $ok .= "-"; # protocol not checked 1416*6236dae4SAndroid Build Coastguard Worker } 1417*6236dae4SAndroid Build Coastguard Worker 1418*6236dae4SAndroid Build Coastguard Worker my %replyattr = getpartattr("reply", "data"); 1419*6236dae4SAndroid Build Coastguard Worker my @reply; 1420*6236dae4SAndroid Build Coastguard Worker if (partexists("reply", "datacheck")) { 1421*6236dae4SAndroid Build Coastguard Worker for my $partsuffix (('', '1', '2', '3', '4')) { 1422*6236dae4SAndroid Build Coastguard Worker my @replycheckpart = getpart("reply", "datacheck".$partsuffix); 1423*6236dae4SAndroid Build Coastguard Worker if(@replycheckpart) { 1424*6236dae4SAndroid Build Coastguard Worker my %replycheckpartattr = getpartattr("reply", "datacheck".$partsuffix); 1425*6236dae4SAndroid Build Coastguard Worker # get the mode attribute 1426*6236dae4SAndroid Build Coastguard Worker my $filemode=$replycheckpartattr{'mode'}; 1427*6236dae4SAndroid Build Coastguard Worker if($filemode && ($filemode eq "text")) { 1428*6236dae4SAndroid Build Coastguard Worker normalize_text(\@replycheckpart); 1429*6236dae4SAndroid Build Coastguard Worker } 1430*6236dae4SAndroid Build Coastguard Worker if($replycheckpartattr{'nonewline'}) { 1431*6236dae4SAndroid Build Coastguard Worker # Yes, we must cut off the final newline from the final line 1432*6236dae4SAndroid Build Coastguard Worker # of the datacheck 1433*6236dae4SAndroid Build Coastguard Worker chomp($replycheckpart[-1]); 1434*6236dae4SAndroid Build Coastguard Worker } 1435*6236dae4SAndroid Build Coastguard Worker if($replycheckpartattr{'crlf'} || 1436*6236dae4SAndroid Build Coastguard Worker ($feature{"hyper"} && ($keywords{"HTTP"} 1437*6236dae4SAndroid Build Coastguard Worker || $keywords{"HTTPS"}))) { 1438*6236dae4SAndroid Build Coastguard Worker subnewlines(0, \$_) for @replycheckpart; 1439*6236dae4SAndroid Build Coastguard Worker } 1440*6236dae4SAndroid Build Coastguard Worker push(@reply, @replycheckpart); 1441*6236dae4SAndroid Build Coastguard Worker } 1442*6236dae4SAndroid Build Coastguard Worker } 1443*6236dae4SAndroid Build Coastguard Worker } 1444*6236dae4SAndroid Build Coastguard Worker else { 1445*6236dae4SAndroid Build Coastguard Worker # check against the data section 1446*6236dae4SAndroid Build Coastguard Worker @reply = getpart("reply", "data"); 1447*6236dae4SAndroid Build Coastguard Worker if(@reply) { 1448*6236dae4SAndroid Build Coastguard Worker if($replyattr{'nonewline'}) { 1449*6236dae4SAndroid Build Coastguard Worker # cut off the final newline from the final line of the data 1450*6236dae4SAndroid Build Coastguard Worker chomp($reply[-1]); 1451*6236dae4SAndroid Build Coastguard Worker } 1452*6236dae4SAndroid Build Coastguard Worker } 1453*6236dae4SAndroid Build Coastguard Worker # get the mode attribute 1454*6236dae4SAndroid Build Coastguard Worker my $filemode=$replyattr{'mode'}; 1455*6236dae4SAndroid Build Coastguard Worker if($filemode && ($filemode eq "text")) { 1456*6236dae4SAndroid Build Coastguard Worker normalize_text(\@reply); 1457*6236dae4SAndroid Build Coastguard Worker } 1458*6236dae4SAndroid Build Coastguard Worker if($replyattr{'crlf'} || 1459*6236dae4SAndroid Build Coastguard Worker ($feature{"hyper"} && ($keywords{"HTTP"} 1460*6236dae4SAndroid Build Coastguard Worker || $keywords{"HTTPS"}))) { 1461*6236dae4SAndroid Build Coastguard Worker subnewlines(0, \$_) for @reply; 1462*6236dae4SAndroid Build Coastguard Worker } 1463*6236dae4SAndroid Build Coastguard Worker } 1464*6236dae4SAndroid Build Coastguard Worker 1465*6236dae4SAndroid Build Coastguard Worker if(!$replyattr{'nocheck'} && (@reply || $replyattr{'sendzero'})) { 1466*6236dae4SAndroid Build Coastguard Worker # verify the received data 1467*6236dae4SAndroid Build Coastguard Worker my @out = loadarray($CURLOUT); 1468*6236dae4SAndroid Build Coastguard Worker 1469*6236dae4SAndroid Build Coastguard Worker # get the mode attribute 1470*6236dae4SAndroid Build Coastguard Worker my $filemode=$replyattr{'mode'}; 1471*6236dae4SAndroid Build Coastguard Worker if($filemode && ($filemode eq "text")) { 1472*6236dae4SAndroid Build Coastguard Worker normalize_text(\@out); 1473*6236dae4SAndroid Build Coastguard Worker } 1474*6236dae4SAndroid Build Coastguard Worker $res = compare($runnerid, $testnum, $testname, "data", \@out, \@reply); 1475*6236dae4SAndroid Build Coastguard Worker if ($res) { 1476*6236dae4SAndroid Build Coastguard Worker return -1; 1477*6236dae4SAndroid Build Coastguard Worker } 1478*6236dae4SAndroid Build Coastguard Worker $ok .= "d"; 1479*6236dae4SAndroid Build Coastguard Worker } 1480*6236dae4SAndroid Build Coastguard Worker else { 1481*6236dae4SAndroid Build Coastguard Worker $ok .= "-"; # data not checked 1482*6236dae4SAndroid Build Coastguard Worker } 1483*6236dae4SAndroid Build Coastguard Worker 1484*6236dae4SAndroid Build Coastguard Worker # if this section exists, we verify upload 1485*6236dae4SAndroid Build Coastguard Worker my @upload = getpart("verify", "upload"); 1486*6236dae4SAndroid Build Coastguard Worker if(@upload) { 1487*6236dae4SAndroid Build Coastguard Worker my %hash = getpartattr("verify", "upload"); 1488*6236dae4SAndroid Build Coastguard Worker if($hash{'nonewline'}) { 1489*6236dae4SAndroid Build Coastguard Worker # cut off the final newline from the final line of the upload data 1490*6236dae4SAndroid Build Coastguard Worker chomp($upload[-1]); 1491*6236dae4SAndroid Build Coastguard Worker } 1492*6236dae4SAndroid Build Coastguard Worker for my $line (@upload) { 1493*6236dae4SAndroid Build Coastguard Worker subbase64(\$line); 1494*6236dae4SAndroid Build Coastguard Worker subsha256base64file(\$line); 1495*6236dae4SAndroid Build Coastguard Worker substrippemfile(\$line); 1496*6236dae4SAndroid Build Coastguard Worker } 1497*6236dae4SAndroid Build Coastguard Worker 1498*6236dae4SAndroid Build Coastguard Worker # verify uploaded data 1499*6236dae4SAndroid Build Coastguard Worker my @out = loadarray("$logdir/upload.$testnum"); 1500*6236dae4SAndroid Build Coastguard Worker for my $strip (@strippart) { 1501*6236dae4SAndroid Build Coastguard Worker chomp $strip; 1502*6236dae4SAndroid Build Coastguard Worker for(@out) { 1503*6236dae4SAndroid Build Coastguard Worker eval $strip; 1504*6236dae4SAndroid Build Coastguard Worker } 1505*6236dae4SAndroid Build Coastguard Worker } 1506*6236dae4SAndroid Build Coastguard Worker if($hash{'crlf'}) { 1507*6236dae4SAndroid Build Coastguard Worker subnewlines(1, \$_) for @upload; 1508*6236dae4SAndroid Build Coastguard Worker } 1509*6236dae4SAndroid Build Coastguard Worker if($hash{'nonewline'}) { 1510*6236dae4SAndroid Build Coastguard Worker # Yes, we must cut off the final newline from the final line 1511*6236dae4SAndroid Build Coastguard Worker # of the upload data 1512*6236dae4SAndroid Build Coastguard Worker chomp($upload[-1]); 1513*6236dae4SAndroid Build Coastguard Worker } 1514*6236dae4SAndroid Build Coastguard Worker 1515*6236dae4SAndroid Build Coastguard Worker $res = compare($runnerid, $testnum, $testname, "upload", \@out, \@upload); 1516*6236dae4SAndroid Build Coastguard Worker if ($res) { 1517*6236dae4SAndroid Build Coastguard Worker return -1; 1518*6236dae4SAndroid Build Coastguard Worker } 1519*6236dae4SAndroid Build Coastguard Worker $ok .= "u"; 1520*6236dae4SAndroid Build Coastguard Worker } 1521*6236dae4SAndroid Build Coastguard Worker else { 1522*6236dae4SAndroid Build Coastguard Worker $ok .= "-"; # upload not checked 1523*6236dae4SAndroid Build Coastguard Worker } 1524*6236dae4SAndroid Build Coastguard Worker 1525*6236dae4SAndroid Build Coastguard Worker # this is the valid protocol blurb curl should generate to a proxy 1526*6236dae4SAndroid Build Coastguard Worker my @proxyprot = getpart("verify", "proxy"); 1527*6236dae4SAndroid Build Coastguard Worker if(@proxyprot) { 1528*6236dae4SAndroid Build Coastguard Worker # Verify the sent proxy request 1529*6236dae4SAndroid Build Coastguard Worker # check if there's any attributes on the verify/protocol section 1530*6236dae4SAndroid Build Coastguard Worker my %hash = getpartattr("verify", "proxy"); 1531*6236dae4SAndroid Build Coastguard Worker 1532*6236dae4SAndroid Build Coastguard Worker if($hash{'nonewline'}) { 1533*6236dae4SAndroid Build Coastguard Worker # Yes, we must cut off the final newline from the final line 1534*6236dae4SAndroid Build Coastguard Worker # of the protocol data 1535*6236dae4SAndroid Build Coastguard Worker chomp($proxyprot[-1]); 1536*6236dae4SAndroid Build Coastguard Worker } 1537*6236dae4SAndroid Build Coastguard Worker 1538*6236dae4SAndroid Build Coastguard Worker my @out = loadarray("$logdir/$PROXYIN"); 1539*6236dae4SAndroid Build Coastguard Worker for(@strip) { 1540*6236dae4SAndroid Build Coastguard Worker # strip off all lines that match the patterns from both arrays 1541*6236dae4SAndroid Build Coastguard Worker chomp $_; 1542*6236dae4SAndroid Build Coastguard Worker @out = striparray( $_, \@out); 1543*6236dae4SAndroid Build Coastguard Worker @proxyprot= striparray( $_, \@proxyprot); 1544*6236dae4SAndroid Build Coastguard Worker } 1545*6236dae4SAndroid Build Coastguard Worker 1546*6236dae4SAndroid Build Coastguard Worker for my $strip (@strippart) { 1547*6236dae4SAndroid Build Coastguard Worker chomp $strip; 1548*6236dae4SAndroid Build Coastguard Worker for(@out) { 1549*6236dae4SAndroid Build Coastguard Worker eval $strip; 1550*6236dae4SAndroid Build Coastguard Worker } 1551*6236dae4SAndroid Build Coastguard Worker } 1552*6236dae4SAndroid Build Coastguard Worker 1553*6236dae4SAndroid Build Coastguard Worker if($hash{'crlf'} || 1554*6236dae4SAndroid Build Coastguard Worker ($feature{"hyper"} && ($keywords{"HTTP"} || $keywords{"HTTPS"}))) { 1555*6236dae4SAndroid Build Coastguard Worker subnewlines(0, \$_) for @proxyprot; 1556*6236dae4SAndroid Build Coastguard Worker } 1557*6236dae4SAndroid Build Coastguard Worker 1558*6236dae4SAndroid Build Coastguard Worker $res = compare($runnerid, $testnum, $testname, "proxy", \@out, \@proxyprot); 1559*6236dae4SAndroid Build Coastguard Worker if($res) { 1560*6236dae4SAndroid Build Coastguard Worker return -1; 1561*6236dae4SAndroid Build Coastguard Worker } 1562*6236dae4SAndroid Build Coastguard Worker 1563*6236dae4SAndroid Build Coastguard Worker $ok .= "P"; 1564*6236dae4SAndroid Build Coastguard Worker 1565*6236dae4SAndroid Build Coastguard Worker } 1566*6236dae4SAndroid Build Coastguard Worker else { 1567*6236dae4SAndroid Build Coastguard Worker $ok .= "-"; # proxy not checked 1568*6236dae4SAndroid Build Coastguard Worker } 1569*6236dae4SAndroid Build Coastguard Worker 1570*6236dae4SAndroid Build Coastguard Worker my $outputok; 1571*6236dae4SAndroid Build Coastguard Worker for my $partsuffix (('', '1', '2', '3', '4')) { 1572*6236dae4SAndroid Build Coastguard Worker my @outfile=getpart("verify", "file".$partsuffix); 1573*6236dae4SAndroid Build Coastguard Worker if(@outfile || partexists("verify", "file".$partsuffix) ) { 1574*6236dae4SAndroid Build Coastguard Worker # we're supposed to verify a dynamically generated file! 1575*6236dae4SAndroid Build Coastguard Worker my %hash = getpartattr("verify", "file".$partsuffix); 1576*6236dae4SAndroid Build Coastguard Worker 1577*6236dae4SAndroid Build Coastguard Worker my $filename=$hash{'name'}; 1578*6236dae4SAndroid Build Coastguard Worker if(!$filename) { 1579*6236dae4SAndroid Build Coastguard Worker logmsg " $testnum: IGNORED: section verify=>file$partsuffix ". 1580*6236dae4SAndroid Build Coastguard Worker "has no name attribute\n"; 1581*6236dae4SAndroid Build Coastguard Worker if (runnerac_stopservers($runnerid)) { 1582*6236dae4SAndroid Build Coastguard Worker logmsg "ERROR: runner $runnerid seems to have died\n"; 1583*6236dae4SAndroid Build Coastguard Worker } else { 1584*6236dae4SAndroid Build Coastguard Worker 1585*6236dae4SAndroid Build Coastguard Worker # TODO: this is a blocking call that will stall the controller, 1586*6236dae4SAndroid Build Coastguard Worker if($verbose) { 1587*6236dae4SAndroid Build Coastguard Worker logmsg "WARNING: blocking call in async function\n"; 1588*6236dae4SAndroid Build Coastguard Worker } 1589*6236dae4SAndroid Build Coastguard Worker # but this error condition should never happen except during 1590*6236dae4SAndroid Build Coastguard Worker # development. 1591*6236dae4SAndroid Build Coastguard Worker my ($rid, $unexpected, $logs) = runnerar($runnerid); 1592*6236dae4SAndroid Build Coastguard Worker if(!$rid) { 1593*6236dae4SAndroid Build Coastguard Worker logmsg "ERROR: runner $runnerid seems to have died\n"; 1594*6236dae4SAndroid Build Coastguard Worker } else { 1595*6236dae4SAndroid Build Coastguard Worker logmsg $logs; 1596*6236dae4SAndroid Build Coastguard Worker } 1597*6236dae4SAndroid Build Coastguard Worker } 1598*6236dae4SAndroid Build Coastguard Worker # timestamp test result verification end 1599*6236dae4SAndroid Build Coastguard Worker $timevrfyend{$testnum} = Time::HiRes::time(); 1600*6236dae4SAndroid Build Coastguard Worker return -1; 1601*6236dae4SAndroid Build Coastguard Worker } 1602*6236dae4SAndroid Build Coastguard Worker my @generated=loadarray($filename); 1603*6236dae4SAndroid Build Coastguard Worker 1604*6236dae4SAndroid Build Coastguard Worker # what parts to cut off from the file 1605*6236dae4SAndroid Build Coastguard Worker my @stripfilepar = getpart("verify", "stripfile".$partsuffix); 1606*6236dae4SAndroid Build Coastguard Worker 1607*6236dae4SAndroid Build Coastguard Worker my $filemode=$hash{'mode'}; 1608*6236dae4SAndroid Build Coastguard Worker if($filemode && ($filemode eq "text")) { 1609*6236dae4SAndroid Build Coastguard Worker normalize_text(\@outfile); 1610*6236dae4SAndroid Build Coastguard Worker normalize_text(\@generated); 1611*6236dae4SAndroid Build Coastguard Worker } 1612*6236dae4SAndroid Build Coastguard Worker if($hash{'crlf'} || 1613*6236dae4SAndroid Build Coastguard Worker ($feature{"hyper"} && ($keywords{"HTTP"} 1614*6236dae4SAndroid Build Coastguard Worker || $keywords{"HTTPS"}))) { 1615*6236dae4SAndroid Build Coastguard Worker subnewlines(0, \$_) for @outfile; 1616*6236dae4SAndroid Build Coastguard Worker } 1617*6236dae4SAndroid Build Coastguard Worker 1618*6236dae4SAndroid Build Coastguard Worker for my $strip (@stripfilepar) { 1619*6236dae4SAndroid Build Coastguard Worker chomp $strip; 1620*6236dae4SAndroid Build Coastguard Worker my @newgen; 1621*6236dae4SAndroid Build Coastguard Worker for(@generated) { 1622*6236dae4SAndroid Build Coastguard Worker eval $strip; 1623*6236dae4SAndroid Build Coastguard Worker if($_) { 1624*6236dae4SAndroid Build Coastguard Worker push @newgen, $_; 1625*6236dae4SAndroid Build Coastguard Worker } 1626*6236dae4SAndroid Build Coastguard Worker } 1627*6236dae4SAndroid Build Coastguard Worker # this is to get rid of array entries that vanished (zero 1628*6236dae4SAndroid Build Coastguard Worker # length) because of replacements 1629*6236dae4SAndroid Build Coastguard Worker @generated = @newgen; 1630*6236dae4SAndroid Build Coastguard Worker } 1631*6236dae4SAndroid Build Coastguard Worker 1632*6236dae4SAndroid Build Coastguard Worker if($hash{'nonewline'}) { 1633*6236dae4SAndroid Build Coastguard Worker # cut off the final newline from the final line of the 1634*6236dae4SAndroid Build Coastguard Worker # output data 1635*6236dae4SAndroid Build Coastguard Worker chomp($outfile[-1]); 1636*6236dae4SAndroid Build Coastguard Worker } 1637*6236dae4SAndroid Build Coastguard Worker 1638*6236dae4SAndroid Build Coastguard Worker $res = compare($runnerid, $testnum, $testname, "output ($filename)", 1639*6236dae4SAndroid Build Coastguard Worker \@generated, \@outfile); 1640*6236dae4SAndroid Build Coastguard Worker if($res) { 1641*6236dae4SAndroid Build Coastguard Worker return -1; 1642*6236dae4SAndroid Build Coastguard Worker } 1643*6236dae4SAndroid Build Coastguard Worker 1644*6236dae4SAndroid Build Coastguard Worker $outputok = 1; # output checked 1645*6236dae4SAndroid Build Coastguard Worker } 1646*6236dae4SAndroid Build Coastguard Worker } 1647*6236dae4SAndroid Build Coastguard Worker $ok .= ($outputok) ? "o" : "-"; # output checked or not 1648*6236dae4SAndroid Build Coastguard Worker 1649*6236dae4SAndroid Build Coastguard Worker # verify SOCKS proxy details 1650*6236dae4SAndroid Build Coastguard Worker my @socksprot = getpart("verify", "socks"); 1651*6236dae4SAndroid Build Coastguard Worker if(@socksprot) { 1652*6236dae4SAndroid Build Coastguard Worker # Verify the sent SOCKS proxy details 1653*6236dae4SAndroid Build Coastguard Worker my @out = loadarray("$logdir/$SOCKSIN"); 1654*6236dae4SAndroid Build Coastguard Worker $res = compare($runnerid, $testnum, $testname, "socks", \@out, \@socksprot); 1655*6236dae4SAndroid Build Coastguard Worker if($res) { 1656*6236dae4SAndroid Build Coastguard Worker return -1; 1657*6236dae4SAndroid Build Coastguard Worker } 1658*6236dae4SAndroid Build Coastguard Worker } 1659*6236dae4SAndroid Build Coastguard Worker 1660*6236dae4SAndroid Build Coastguard Worker # accept multiple comma-separated error codes 1661*6236dae4SAndroid Build Coastguard Worker my @splerr = split(/ *, */, $errorcode); 1662*6236dae4SAndroid Build Coastguard Worker my $errok; 1663*6236dae4SAndroid Build Coastguard Worker foreach my $e (@splerr) { 1664*6236dae4SAndroid Build Coastguard Worker if($e == $cmdres) { 1665*6236dae4SAndroid Build Coastguard Worker # a fine error code 1666*6236dae4SAndroid Build Coastguard Worker $errok = 1; 1667*6236dae4SAndroid Build Coastguard Worker last; 1668*6236dae4SAndroid Build Coastguard Worker } 1669*6236dae4SAndroid Build Coastguard Worker } 1670*6236dae4SAndroid Build Coastguard Worker 1671*6236dae4SAndroid Build Coastguard Worker if($errok) { 1672*6236dae4SAndroid Build Coastguard Worker $ok .= "e"; 1673*6236dae4SAndroid Build Coastguard Worker } 1674*6236dae4SAndroid Build Coastguard Worker else { 1675*6236dae4SAndroid Build Coastguard Worker if(!$short) { 1676*6236dae4SAndroid Build Coastguard Worker logmsg sprintf("\n%s returned $cmdres, when expecting %s\n", 1677*6236dae4SAndroid Build Coastguard Worker (!$tool)?"curl":$tool, $errorcode); 1678*6236dae4SAndroid Build Coastguard Worker } 1679*6236dae4SAndroid Build Coastguard Worker logmsg " $testnum: exit FAILED\n"; 1680*6236dae4SAndroid Build Coastguard Worker # timestamp test result verification end 1681*6236dae4SAndroid Build Coastguard Worker $timevrfyend{$testnum} = Time::HiRes::time(); 1682*6236dae4SAndroid Build Coastguard Worker return -1; 1683*6236dae4SAndroid Build Coastguard Worker } 1684*6236dae4SAndroid Build Coastguard Worker 1685*6236dae4SAndroid Build Coastguard Worker if($feature{"TrackMemory"}) { 1686*6236dae4SAndroid Build Coastguard Worker if(! -f "$logdir/$MEMDUMP") { 1687*6236dae4SAndroid Build Coastguard Worker my %cmdhash = getpartattr("client", "command"); 1688*6236dae4SAndroid Build Coastguard Worker my $cmdtype = $cmdhash{'type'} || "default"; 1689*6236dae4SAndroid Build Coastguard Worker logmsg "\n** ALERT! memory tracking with no output file?\n" 1690*6236dae4SAndroid Build Coastguard Worker if(!$cmdtype eq "perl"); 1691*6236dae4SAndroid Build Coastguard Worker $ok .= "-"; # problem with memory checking 1692*6236dae4SAndroid Build Coastguard Worker } 1693*6236dae4SAndroid Build Coastguard Worker else { 1694*6236dae4SAndroid Build Coastguard Worker my @memdata=`$memanalyze "$logdir/$MEMDUMP"`; 1695*6236dae4SAndroid Build Coastguard Worker my $leak=0; 1696*6236dae4SAndroid Build Coastguard Worker for(@memdata) { 1697*6236dae4SAndroid Build Coastguard Worker if($_ ne "") { 1698*6236dae4SAndroid Build Coastguard Worker # well it could be other memory problems as well, but 1699*6236dae4SAndroid Build Coastguard Worker # we call it leak for short here 1700*6236dae4SAndroid Build Coastguard Worker $leak=1; 1701*6236dae4SAndroid Build Coastguard Worker } 1702*6236dae4SAndroid Build Coastguard Worker } 1703*6236dae4SAndroid Build Coastguard Worker if($leak) { 1704*6236dae4SAndroid Build Coastguard Worker logmsg "\n** MEMORY FAILURE\n"; 1705*6236dae4SAndroid Build Coastguard Worker logmsg @memdata; 1706*6236dae4SAndroid Build Coastguard Worker # timestamp test result verification end 1707*6236dae4SAndroid Build Coastguard Worker $timevrfyend{$testnum} = Time::HiRes::time(); 1708*6236dae4SAndroid Build Coastguard Worker return -1; 1709*6236dae4SAndroid Build Coastguard Worker } 1710*6236dae4SAndroid Build Coastguard Worker else { 1711*6236dae4SAndroid Build Coastguard Worker $ok .= "m"; 1712*6236dae4SAndroid Build Coastguard Worker } 1713*6236dae4SAndroid Build Coastguard Worker } 1714*6236dae4SAndroid Build Coastguard Worker } 1715*6236dae4SAndroid Build Coastguard Worker else { 1716*6236dae4SAndroid Build Coastguard Worker $ok .= "-"; # memory not checked 1717*6236dae4SAndroid Build Coastguard Worker } 1718*6236dae4SAndroid Build Coastguard Worker 1719*6236dae4SAndroid Build Coastguard Worker my @notexists = getpart("verify", "notexists"); 1720*6236dae4SAndroid Build Coastguard Worker if(@notexists) { 1721*6236dae4SAndroid Build Coastguard Worker # a list of directory entries that must not exist 1722*6236dae4SAndroid Build Coastguard Worker my $err; 1723*6236dae4SAndroid Build Coastguard Worker while (@notexists) { 1724*6236dae4SAndroid Build Coastguard Worker my $fname = shift @notexists; 1725*6236dae4SAndroid Build Coastguard Worker chomp $fname; 1726*6236dae4SAndroid Build Coastguard Worker if (-e $fname) { 1727*6236dae4SAndroid Build Coastguard Worker logmsg "Found '$fname' when not supposed to exist.\n"; 1728*6236dae4SAndroid Build Coastguard Worker $err++; 1729*6236dae4SAndroid Build Coastguard Worker } 1730*6236dae4SAndroid Build Coastguard Worker elsif($verbose) { 1731*6236dae4SAndroid Build Coastguard Worker logmsg "Found '$fname' confirmed to not exist.\n"; 1732*6236dae4SAndroid Build Coastguard Worker } 1733*6236dae4SAndroid Build Coastguard Worker } 1734*6236dae4SAndroid Build Coastguard Worker if($err) { 1735*6236dae4SAndroid Build Coastguard Worker return -1; 1736*6236dae4SAndroid Build Coastguard Worker } 1737*6236dae4SAndroid Build Coastguard Worker } 1738*6236dae4SAndroid Build Coastguard Worker if($valgrind) { 1739*6236dae4SAndroid Build Coastguard Worker if($usedvalgrind) { 1740*6236dae4SAndroid Build Coastguard Worker if(!opendir(DIR, "$logdir")) { 1741*6236dae4SAndroid Build Coastguard Worker logmsg "ERROR: unable to read $logdir\n"; 1742*6236dae4SAndroid Build Coastguard Worker # timestamp test result verification end 1743*6236dae4SAndroid Build Coastguard Worker $timevrfyend{$testnum} = Time::HiRes::time(); 1744*6236dae4SAndroid Build Coastguard Worker return -1; 1745*6236dae4SAndroid Build Coastguard Worker } 1746*6236dae4SAndroid Build Coastguard Worker my @files = readdir(DIR); 1747*6236dae4SAndroid Build Coastguard Worker closedir(DIR); 1748*6236dae4SAndroid Build Coastguard Worker my $vgfile; 1749*6236dae4SAndroid Build Coastguard Worker foreach my $file (@files) { 1750*6236dae4SAndroid Build Coastguard Worker if($file =~ /^valgrind$testnum(\..*|)$/) { 1751*6236dae4SAndroid Build Coastguard Worker $vgfile = $file; 1752*6236dae4SAndroid Build Coastguard Worker last; 1753*6236dae4SAndroid Build Coastguard Worker } 1754*6236dae4SAndroid Build Coastguard Worker } 1755*6236dae4SAndroid Build Coastguard Worker if(!$vgfile) { 1756*6236dae4SAndroid Build Coastguard Worker logmsg "ERROR: valgrind log file missing for test $testnum\n"; 1757*6236dae4SAndroid Build Coastguard Worker # timestamp test result verification end 1758*6236dae4SAndroid Build Coastguard Worker $timevrfyend{$testnum} = Time::HiRes::time(); 1759*6236dae4SAndroid Build Coastguard Worker return -1; 1760*6236dae4SAndroid Build Coastguard Worker } 1761*6236dae4SAndroid Build Coastguard Worker my @e = valgrindparse("$logdir/$vgfile"); 1762*6236dae4SAndroid Build Coastguard Worker if(@e && $e[0]) { 1763*6236dae4SAndroid Build Coastguard Worker if($automakestyle) { 1764*6236dae4SAndroid Build Coastguard Worker logmsg "FAIL: $testnum - $testname - valgrind\n"; 1765*6236dae4SAndroid Build Coastguard Worker } 1766*6236dae4SAndroid Build Coastguard Worker else { 1767*6236dae4SAndroid Build Coastguard Worker logmsg " valgrind ERROR "; 1768*6236dae4SAndroid Build Coastguard Worker logmsg @e; 1769*6236dae4SAndroid Build Coastguard Worker } 1770*6236dae4SAndroid Build Coastguard Worker # timestamp test result verification end 1771*6236dae4SAndroid Build Coastguard Worker $timevrfyend{$testnum} = Time::HiRes::time(); 1772*6236dae4SAndroid Build Coastguard Worker return -1; 1773*6236dae4SAndroid Build Coastguard Worker } 1774*6236dae4SAndroid Build Coastguard Worker $ok .= "v"; 1775*6236dae4SAndroid Build Coastguard Worker } 1776*6236dae4SAndroid Build Coastguard Worker else { 1777*6236dae4SAndroid Build Coastguard Worker if($verbose) { 1778*6236dae4SAndroid Build Coastguard Worker logmsg " valgrind SKIPPED\n"; 1779*6236dae4SAndroid Build Coastguard Worker } 1780*6236dae4SAndroid Build Coastguard Worker $ok .= "-"; # skipped 1781*6236dae4SAndroid Build Coastguard Worker } 1782*6236dae4SAndroid Build Coastguard Worker } 1783*6236dae4SAndroid Build Coastguard Worker else { 1784*6236dae4SAndroid Build Coastguard Worker $ok .= "-"; # valgrind not checked 1785*6236dae4SAndroid Build Coastguard Worker } 1786*6236dae4SAndroid Build Coastguard Worker # add 'E' for event-based 1787*6236dae4SAndroid Build Coastguard Worker $ok .= $run_event_based ? "E" : "-"; 1788*6236dae4SAndroid Build Coastguard Worker 1789*6236dae4SAndroid Build Coastguard Worker logmsg "$ok " if(!$short); 1790*6236dae4SAndroid Build Coastguard Worker 1791*6236dae4SAndroid Build Coastguard Worker # timestamp test result verification end 1792*6236dae4SAndroid Build Coastguard Worker $timevrfyend{$testnum} = Time::HiRes::time(); 1793*6236dae4SAndroid Build Coastguard Worker 1794*6236dae4SAndroid Build Coastguard Worker return 0; 1795*6236dae4SAndroid Build Coastguard Worker} 1796*6236dae4SAndroid Build Coastguard Worker 1797*6236dae4SAndroid Build Coastguard Worker 1798*6236dae4SAndroid Build Coastguard Worker####################################################################### 1799*6236dae4SAndroid Build Coastguard Worker# Report a successful test 1800*6236dae4SAndroid Build Coastguard Workersub singletest_success { 1801*6236dae4SAndroid Build Coastguard Worker my ($testnum, $count, $total, $errorreturncode)=@_; 1802*6236dae4SAndroid Build Coastguard Worker 1803*6236dae4SAndroid Build Coastguard Worker my $sofar= time()-$start; 1804*6236dae4SAndroid Build Coastguard Worker my $esttotal = $sofar/$count * $total; 1805*6236dae4SAndroid Build Coastguard Worker my $estleft = $esttotal - $sofar; 1806*6236dae4SAndroid Build Coastguard Worker my $timeleft=sprintf("remaining: %02d:%02d", 1807*6236dae4SAndroid Build Coastguard Worker $estleft/60, 1808*6236dae4SAndroid Build Coastguard Worker $estleft%60); 1809*6236dae4SAndroid Build Coastguard Worker my $took = $timevrfyend{$testnum} - $timeprepini{$testnum}; 1810*6236dae4SAndroid Build Coastguard Worker my $duration = sprintf("duration: %02d:%02d", 1811*6236dae4SAndroid Build Coastguard Worker $sofar/60, $sofar%60); 1812*6236dae4SAndroid Build Coastguard Worker if(!$automakestyle) { 1813*6236dae4SAndroid Build Coastguard Worker logmsg sprintf("OK (%-3d out of %-3d, %s, took %.3fs, %s)\n", 1814*6236dae4SAndroid Build Coastguard Worker $count, $total, $timeleft, $took, $duration); 1815*6236dae4SAndroid Build Coastguard Worker } 1816*6236dae4SAndroid Build Coastguard Worker else { 1817*6236dae4SAndroid Build Coastguard Worker my $testname= (getpart("client", "name"))[0]; 1818*6236dae4SAndroid Build Coastguard Worker chomp $testname; 1819*6236dae4SAndroid Build Coastguard Worker logmsg "PASS: $testnum - $testname\n"; 1820*6236dae4SAndroid Build Coastguard Worker } 1821*6236dae4SAndroid Build Coastguard Worker 1822*6236dae4SAndroid Build Coastguard Worker if($errorreturncode==2) { 1823*6236dae4SAndroid Build Coastguard Worker # ignored test success 1824*6236dae4SAndroid Build Coastguard Worker $passedign .= "$testnum "; 1825*6236dae4SAndroid Build Coastguard Worker logmsg "Warning: test$testnum result is ignored, but passed!\n"; 1826*6236dae4SAndroid Build Coastguard Worker } 1827*6236dae4SAndroid Build Coastguard Worker} 1828*6236dae4SAndroid Build Coastguard Worker 1829*6236dae4SAndroid Build Coastguard Worker####################################################################### 1830*6236dae4SAndroid Build Coastguard Worker# Run a single specified test case 1831*6236dae4SAndroid Build Coastguard Worker# This is structured as a state machine which changes state after an 1832*6236dae4SAndroid Build Coastguard Worker# asynchronous call is made that awaits a response. The function returns with 1833*6236dae4SAndroid Build Coastguard Worker# an error code and a flag that indicates if the state machine has completed, 1834*6236dae4SAndroid Build Coastguard Worker# which means (if not) the function must be called again once the response has 1835*6236dae4SAndroid Build Coastguard Worker# arrived. 1836*6236dae4SAndroid Build Coastguard Worker# 1837*6236dae4SAndroid Build Coastguard Workersub singletest { 1838*6236dae4SAndroid Build Coastguard Worker my ($runnerid, $testnum, $count, $total)=@_; 1839*6236dae4SAndroid Build Coastguard Worker 1840*6236dae4SAndroid Build Coastguard Worker # start buffering logmsg; stop it on return 1841*6236dae4SAndroid Build Coastguard Worker logmsg_bufferfortest($runnerid); 1842*6236dae4SAndroid Build Coastguard Worker if(!exists $singletest_state{$runnerid}) { 1843*6236dae4SAndroid Build Coastguard Worker # First time in singletest() for this test 1844*6236dae4SAndroid Build Coastguard Worker $singletest_state{$runnerid} = ST_INIT; 1845*6236dae4SAndroid Build Coastguard Worker } 1846*6236dae4SAndroid Build Coastguard Worker 1847*6236dae4SAndroid Build Coastguard Worker if($singletest_state{$runnerid} == ST_INIT) { 1848*6236dae4SAndroid Build Coastguard Worker my $logdir = getrunnerlogdir($runnerid); 1849*6236dae4SAndroid Build Coastguard Worker # first, remove all lingering log & lock files 1850*6236dae4SAndroid Build Coastguard Worker if((!cleardir($logdir) || !cleardir("$logdir/$LOCKDIR")) 1851*6236dae4SAndroid Build Coastguard Worker && $clearlocks) { 1852*6236dae4SAndroid Build Coastguard Worker # On Windows, lock files can't be deleted when the process still 1853*6236dae4SAndroid Build Coastguard Worker # has them open, so kill those processes first 1854*6236dae4SAndroid Build Coastguard Worker if(runnerac_clearlocks($runnerid, "$logdir/$LOCKDIR")) { 1855*6236dae4SAndroid Build Coastguard Worker logmsg "ERROR: runner $runnerid seems to have died\n"; 1856*6236dae4SAndroid Build Coastguard Worker $singletest_state{$runnerid} = ST_INIT; 1857*6236dae4SAndroid Build Coastguard Worker return (-1, 0); 1858*6236dae4SAndroid Build Coastguard Worker } 1859*6236dae4SAndroid Build Coastguard Worker $singletest_state{$runnerid} = ST_CLEARLOCKS; 1860*6236dae4SAndroid Build Coastguard Worker } else { 1861*6236dae4SAndroid Build Coastguard Worker $singletest_state{$runnerid} = ST_INITED; 1862*6236dae4SAndroid Build Coastguard Worker # Recursively call the state machine again because there is no 1863*6236dae4SAndroid Build Coastguard Worker # event expected that would otherwise trigger a new call. 1864*6236dae4SAndroid Build Coastguard Worker return singletest(@_); 1865*6236dae4SAndroid Build Coastguard Worker } 1866*6236dae4SAndroid Build Coastguard Worker 1867*6236dae4SAndroid Build Coastguard Worker } elsif($singletest_state{$runnerid} == ST_CLEARLOCKS) { 1868*6236dae4SAndroid Build Coastguard Worker my ($rid, $logs) = runnerar($runnerid); 1869*6236dae4SAndroid Build Coastguard Worker if(!$rid) { 1870*6236dae4SAndroid Build Coastguard Worker logmsg "ERROR: runner $runnerid seems to have died\n"; 1871*6236dae4SAndroid Build Coastguard Worker $singletest_state{$runnerid} = ST_INIT; 1872*6236dae4SAndroid Build Coastguard Worker return (-1, 0); 1873*6236dae4SAndroid Build Coastguard Worker } 1874*6236dae4SAndroid Build Coastguard Worker logmsg $logs; 1875*6236dae4SAndroid Build Coastguard Worker my $logdir = getrunnerlogdir($runnerid); 1876*6236dae4SAndroid Build Coastguard Worker cleardir($logdir); 1877*6236dae4SAndroid Build Coastguard Worker $singletest_state{$runnerid} = ST_INITED; 1878*6236dae4SAndroid Build Coastguard Worker # Recursively call the state machine again because there is no 1879*6236dae4SAndroid Build Coastguard Worker # event expected that would otherwise trigger a new call. 1880*6236dae4SAndroid Build Coastguard Worker return singletest(@_); 1881*6236dae4SAndroid Build Coastguard Worker 1882*6236dae4SAndroid Build Coastguard Worker } elsif($singletest_state{$runnerid} == ST_INITED) { 1883*6236dae4SAndroid Build Coastguard Worker ################################################################### 1884*6236dae4SAndroid Build Coastguard Worker # Restore environment variables that were modified in a previous run. 1885*6236dae4SAndroid Build Coastguard Worker # Test definition may instruct to (un)set environment vars. 1886*6236dae4SAndroid Build Coastguard Worker # This is done this early so that leftover variables don't affect 1887*6236dae4SAndroid Build Coastguard Worker # starting servers or CI registration. 1888*6236dae4SAndroid Build Coastguard Worker # restore_test_env(1); 1889*6236dae4SAndroid Build Coastguard Worker 1890*6236dae4SAndroid Build Coastguard Worker ################################################################### 1891*6236dae4SAndroid Build Coastguard Worker # Load test file so CI registration can get the right data before the 1892*6236dae4SAndroid Build Coastguard Worker # runner is called 1893*6236dae4SAndroid Build Coastguard Worker loadtest("${TESTDIR}/test${testnum}"); 1894*6236dae4SAndroid Build Coastguard Worker 1895*6236dae4SAndroid Build Coastguard Worker ################################################################### 1896*6236dae4SAndroid Build Coastguard Worker # Register the test case with the CI environment 1897*6236dae4SAndroid Build Coastguard Worker citest_starttest($testnum); 1898*6236dae4SAndroid Build Coastguard Worker 1899*6236dae4SAndroid Build Coastguard Worker if(runnerac_test_preprocess($runnerid, $testnum)) { 1900*6236dae4SAndroid Build Coastguard Worker logmsg "ERROR: runner $runnerid seems to have died\n"; 1901*6236dae4SAndroid Build Coastguard Worker $singletest_state{$runnerid} = ST_INIT; 1902*6236dae4SAndroid Build Coastguard Worker return (-1, 0); 1903*6236dae4SAndroid Build Coastguard Worker } 1904*6236dae4SAndroid Build Coastguard Worker $singletest_state{$runnerid} = ST_PREPROCESS; 1905*6236dae4SAndroid Build Coastguard Worker 1906*6236dae4SAndroid Build Coastguard Worker } elsif($singletest_state{$runnerid} == ST_PREPROCESS) { 1907*6236dae4SAndroid Build Coastguard Worker my ($rid, $why, $error, $logs, $testtimings) = runnerar($runnerid); 1908*6236dae4SAndroid Build Coastguard Worker if(!$rid) { 1909*6236dae4SAndroid Build Coastguard Worker logmsg "ERROR: runner $runnerid seems to have died\n"; 1910*6236dae4SAndroid Build Coastguard Worker $singletest_state{$runnerid} = ST_INIT; 1911*6236dae4SAndroid Build Coastguard Worker return (-1, 0); 1912*6236dae4SAndroid Build Coastguard Worker } 1913*6236dae4SAndroid Build Coastguard Worker logmsg $logs; 1914*6236dae4SAndroid Build Coastguard Worker updatetesttimings($testnum, %$testtimings); 1915*6236dae4SAndroid Build Coastguard Worker if($error == -2) { 1916*6236dae4SAndroid Build Coastguard Worker if($postmortem) { 1917*6236dae4SAndroid Build Coastguard Worker # Error indicates an actual problem starting the server, so 1918*6236dae4SAndroid Build Coastguard Worker # display the server logs 1919*6236dae4SAndroid Build Coastguard Worker displaylogs($rid, $testnum); 1920*6236dae4SAndroid Build Coastguard Worker } 1921*6236dae4SAndroid Build Coastguard Worker } 1922*6236dae4SAndroid Build Coastguard Worker 1923*6236dae4SAndroid Build Coastguard Worker ####################################################################### 1924*6236dae4SAndroid Build Coastguard Worker # Load test file for this test number 1925*6236dae4SAndroid Build Coastguard Worker my $logdir = getrunnerlogdir($runnerid); 1926*6236dae4SAndroid Build Coastguard Worker loadtest("${logdir}/test${testnum}"); 1927*6236dae4SAndroid Build Coastguard Worker 1928*6236dae4SAndroid Build Coastguard Worker ####################################################################### 1929*6236dae4SAndroid Build Coastguard Worker # Print the test name and count tests 1930*6236dae4SAndroid Build Coastguard Worker $error = singletest_count($testnum, $why); 1931*6236dae4SAndroid Build Coastguard Worker if($error) { 1932*6236dae4SAndroid Build Coastguard Worker # Submit the test case result with the CI environment 1933*6236dae4SAndroid Build Coastguard Worker citest_finishtest($testnum, $error); 1934*6236dae4SAndroid Build Coastguard Worker $singletest_state{$runnerid} = ST_INIT; 1935*6236dae4SAndroid Build Coastguard Worker logmsg singletest_dumplogs(); 1936*6236dae4SAndroid Build Coastguard Worker return ($error, 0); 1937*6236dae4SAndroid Build Coastguard Worker } 1938*6236dae4SAndroid Build Coastguard Worker 1939*6236dae4SAndroid Build Coastguard Worker ####################################################################### 1940*6236dae4SAndroid Build Coastguard Worker # Execute this test number 1941*6236dae4SAndroid Build Coastguard Worker my $cmdres; 1942*6236dae4SAndroid Build Coastguard Worker my $CURLOUT; 1943*6236dae4SAndroid Build Coastguard Worker my $tool; 1944*6236dae4SAndroid Build Coastguard Worker my $usedvalgrind; 1945*6236dae4SAndroid Build Coastguard Worker if(runnerac_test_run($runnerid, $testnum)) { 1946*6236dae4SAndroid Build Coastguard Worker logmsg "ERROR: runner $runnerid seems to have died\n"; 1947*6236dae4SAndroid Build Coastguard Worker $singletest_state{$runnerid} = ST_INIT; 1948*6236dae4SAndroid Build Coastguard Worker return (-1, 0); 1949*6236dae4SAndroid Build Coastguard Worker } 1950*6236dae4SAndroid Build Coastguard Worker $singletest_state{$runnerid} = ST_RUN; 1951*6236dae4SAndroid Build Coastguard Worker 1952*6236dae4SAndroid Build Coastguard Worker } elsif($singletest_state{$runnerid} == ST_RUN) { 1953*6236dae4SAndroid Build Coastguard Worker my ($rid, $error, $logs, $testtimings, $cmdres, $CURLOUT, $tool, $usedvalgrind) = runnerar($runnerid); 1954*6236dae4SAndroid Build Coastguard Worker if(!$rid) { 1955*6236dae4SAndroid Build Coastguard Worker logmsg "ERROR: runner $runnerid seems to have died\n"; 1956*6236dae4SAndroid Build Coastguard Worker $singletest_state{$runnerid} = ST_INIT; 1957*6236dae4SAndroid Build Coastguard Worker return (-1, 0); 1958*6236dae4SAndroid Build Coastguard Worker } 1959*6236dae4SAndroid Build Coastguard Worker logmsg $logs; 1960*6236dae4SAndroid Build Coastguard Worker updatetesttimings($testnum, %$testtimings); 1961*6236dae4SAndroid Build Coastguard Worker if($error == -1) { 1962*6236dae4SAndroid Build Coastguard Worker # no further verification will occur 1963*6236dae4SAndroid Build Coastguard Worker $timevrfyend{$testnum} = Time::HiRes::time(); 1964*6236dae4SAndroid Build Coastguard Worker my $err = ignoreresultcode($testnum); 1965*6236dae4SAndroid Build Coastguard Worker # Submit the test case result with the CI environment 1966*6236dae4SAndroid Build Coastguard Worker citest_finishtest($testnum, $err); 1967*6236dae4SAndroid Build Coastguard Worker $singletest_state{$runnerid} = ST_INIT; 1968*6236dae4SAndroid Build Coastguard Worker logmsg singletest_dumplogs(); 1969*6236dae4SAndroid Build Coastguard Worker # return a test failure, either to be reported or to be ignored 1970*6236dae4SAndroid Build Coastguard Worker return ($err, 0); 1971*6236dae4SAndroid Build Coastguard Worker } 1972*6236dae4SAndroid Build Coastguard Worker elsif($error == -2) { 1973*6236dae4SAndroid Build Coastguard Worker # fill in the missing timings on error 1974*6236dae4SAndroid Build Coastguard Worker timestampskippedevents($testnum); 1975*6236dae4SAndroid Build Coastguard Worker # Submit the test case result with the CI environment 1976*6236dae4SAndroid Build Coastguard Worker citest_finishtest($testnum, $error); 1977*6236dae4SAndroid Build Coastguard Worker $singletest_state{$runnerid} = ST_INIT; 1978*6236dae4SAndroid Build Coastguard Worker logmsg singletest_dumplogs(); 1979*6236dae4SAndroid Build Coastguard Worker return ($error, 0); 1980*6236dae4SAndroid Build Coastguard Worker } 1981*6236dae4SAndroid Build Coastguard Worker elsif($error > 0) { 1982*6236dae4SAndroid Build Coastguard Worker # no further verification will occur 1983*6236dae4SAndroid Build Coastguard Worker $timevrfyend{$testnum} = Time::HiRes::time(); 1984*6236dae4SAndroid Build Coastguard Worker # Submit the test case result with the CI environment 1985*6236dae4SAndroid Build Coastguard Worker citest_finishtest($testnum, $error); 1986*6236dae4SAndroid Build Coastguard Worker $singletest_state{$runnerid} = ST_INIT; 1987*6236dae4SAndroid Build Coastguard Worker logmsg singletest_dumplogs(); 1988*6236dae4SAndroid Build Coastguard Worker return ($error, 0); 1989*6236dae4SAndroid Build Coastguard Worker } 1990*6236dae4SAndroid Build Coastguard Worker 1991*6236dae4SAndroid Build Coastguard Worker ####################################################################### 1992*6236dae4SAndroid Build Coastguard Worker # Verify that the test succeeded 1993*6236dae4SAndroid Build Coastguard Worker # 1994*6236dae4SAndroid Build Coastguard Worker # Load test file for this test number 1995*6236dae4SAndroid Build Coastguard Worker my $logdir = getrunnerlogdir($runnerid); 1996*6236dae4SAndroid Build Coastguard Worker loadtest("${logdir}/test${testnum}"); 1997*6236dae4SAndroid Build Coastguard Worker readtestkeywords(); 1998*6236dae4SAndroid Build Coastguard Worker 1999*6236dae4SAndroid Build Coastguard Worker $error = singletest_check($runnerid, $testnum, $cmdres, $CURLOUT, $tool, $usedvalgrind); 2000*6236dae4SAndroid Build Coastguard Worker if($error == -1) { 2001*6236dae4SAndroid Build Coastguard Worker my $err = ignoreresultcode($testnum); 2002*6236dae4SAndroid Build Coastguard Worker # Submit the test case result with the CI environment 2003*6236dae4SAndroid Build Coastguard Worker citest_finishtest($testnum, $err); 2004*6236dae4SAndroid Build Coastguard Worker $singletest_state{$runnerid} = ST_INIT; 2005*6236dae4SAndroid Build Coastguard Worker logmsg singletest_dumplogs(); 2006*6236dae4SAndroid Build Coastguard Worker # return a test failure, either to be reported or to be ignored 2007*6236dae4SAndroid Build Coastguard Worker return ($err, 0); 2008*6236dae4SAndroid Build Coastguard Worker } 2009*6236dae4SAndroid Build Coastguard Worker elsif($error == -2) { 2010*6236dae4SAndroid Build Coastguard Worker # torture test; there is no verification, so the run result holds the 2011*6236dae4SAndroid Build Coastguard Worker # test success code 2012*6236dae4SAndroid Build Coastguard Worker # Submit the test case result with the CI environment 2013*6236dae4SAndroid Build Coastguard Worker citest_finishtest($testnum, $cmdres); 2014*6236dae4SAndroid Build Coastguard Worker $singletest_state{$runnerid} = ST_INIT; 2015*6236dae4SAndroid Build Coastguard Worker logmsg singletest_dumplogs(); 2016*6236dae4SAndroid Build Coastguard Worker return ($cmdres, 0); 2017*6236dae4SAndroid Build Coastguard Worker } 2018*6236dae4SAndroid Build Coastguard Worker 2019*6236dae4SAndroid Build Coastguard Worker 2020*6236dae4SAndroid Build Coastguard Worker ####################################################################### 2021*6236dae4SAndroid Build Coastguard Worker # Report a successful test 2022*6236dae4SAndroid Build Coastguard Worker singletest_success($testnum, $count, $total, ignoreresultcode($testnum)); 2023*6236dae4SAndroid Build Coastguard Worker 2024*6236dae4SAndroid Build Coastguard Worker # Submit the test case result with the CI environment 2025*6236dae4SAndroid Build Coastguard Worker citest_finishtest($testnum, 0); 2026*6236dae4SAndroid Build Coastguard Worker $singletest_state{$runnerid} = ST_INIT; 2027*6236dae4SAndroid Build Coastguard Worker 2028*6236dae4SAndroid Build Coastguard Worker logmsg singletest_dumplogs(); 2029*6236dae4SAndroid Build Coastguard Worker return (0, 0); # state machine is finished 2030*6236dae4SAndroid Build Coastguard Worker } 2031*6236dae4SAndroid Build Coastguard Worker singletest_unbufferlogs(); 2032*6236dae4SAndroid Build Coastguard Worker return (0, 1); # state machine must be called again on event 2033*6236dae4SAndroid Build Coastguard Worker} 2034*6236dae4SAndroid Build Coastguard Worker 2035*6236dae4SAndroid Build Coastguard Worker####################################################################### 2036*6236dae4SAndroid Build Coastguard Worker# runtimestats displays test-suite run time statistics 2037*6236dae4SAndroid Build Coastguard Worker# 2038*6236dae4SAndroid Build Coastguard Workersub runtimestats { 2039*6236dae4SAndroid Build Coastguard Worker my $lasttest = $_[0]; 2040*6236dae4SAndroid Build Coastguard Worker 2041*6236dae4SAndroid Build Coastguard Worker return if(not $timestats); 2042*6236dae4SAndroid Build Coastguard Worker 2043*6236dae4SAndroid Build Coastguard Worker logmsg "::group::Run Time Stats\n"; 2044*6236dae4SAndroid Build Coastguard Worker 2045*6236dae4SAndroid Build Coastguard Worker logmsg "\nTest suite total running time breakdown per task...\n\n"; 2046*6236dae4SAndroid Build Coastguard Worker 2047*6236dae4SAndroid Build Coastguard Worker my @timesrvr; 2048*6236dae4SAndroid Build Coastguard Worker my @timeprep; 2049*6236dae4SAndroid Build Coastguard Worker my @timetool; 2050*6236dae4SAndroid Build Coastguard Worker my @timelock; 2051*6236dae4SAndroid Build Coastguard Worker my @timevrfy; 2052*6236dae4SAndroid Build Coastguard Worker my @timetest; 2053*6236dae4SAndroid Build Coastguard Worker my $timesrvrtot = 0.0; 2054*6236dae4SAndroid Build Coastguard Worker my $timepreptot = 0.0; 2055*6236dae4SAndroid Build Coastguard Worker my $timetooltot = 0.0; 2056*6236dae4SAndroid Build Coastguard Worker my $timelocktot = 0.0; 2057*6236dae4SAndroid Build Coastguard Worker my $timevrfytot = 0.0; 2058*6236dae4SAndroid Build Coastguard Worker my $timetesttot = 0.0; 2059*6236dae4SAndroid Build Coastguard Worker my $counter; 2060*6236dae4SAndroid Build Coastguard Worker 2061*6236dae4SAndroid Build Coastguard Worker for my $testnum (1 .. $lasttest) { 2062*6236dae4SAndroid Build Coastguard Worker if($timesrvrini{$testnum}) { 2063*6236dae4SAndroid Build Coastguard Worker $timesrvrtot += $timesrvrend{$testnum} - $timesrvrini{$testnum}; 2064*6236dae4SAndroid Build Coastguard Worker $timepreptot += 2065*6236dae4SAndroid Build Coastguard Worker (($timetoolini{$testnum} - $timeprepini{$testnum}) - 2066*6236dae4SAndroid Build Coastguard Worker ($timesrvrend{$testnum} - $timesrvrini{$testnum})); 2067*6236dae4SAndroid Build Coastguard Worker $timetooltot += $timetoolend{$testnum} - $timetoolini{$testnum}; 2068*6236dae4SAndroid Build Coastguard Worker $timelocktot += $timesrvrlog{$testnum} - $timetoolend{$testnum}; 2069*6236dae4SAndroid Build Coastguard Worker $timevrfytot += $timevrfyend{$testnum} - $timesrvrlog{$testnum}; 2070*6236dae4SAndroid Build Coastguard Worker $timetesttot += $timevrfyend{$testnum} - $timeprepini{$testnum}; 2071*6236dae4SAndroid Build Coastguard Worker push @timesrvr, sprintf("%06.3f %04d", 2072*6236dae4SAndroid Build Coastguard Worker $timesrvrend{$testnum} - $timesrvrini{$testnum}, $testnum); 2073*6236dae4SAndroid Build Coastguard Worker push @timeprep, sprintf("%06.3f %04d", 2074*6236dae4SAndroid Build Coastguard Worker ($timetoolini{$testnum} - $timeprepini{$testnum}) - 2075*6236dae4SAndroid Build Coastguard Worker ($timesrvrend{$testnum} - $timesrvrini{$testnum}), $testnum); 2076*6236dae4SAndroid Build Coastguard Worker push @timetool, sprintf("%06.3f %04d", 2077*6236dae4SAndroid Build Coastguard Worker $timetoolend{$testnum} - $timetoolini{$testnum}, $testnum); 2078*6236dae4SAndroid Build Coastguard Worker push @timelock, sprintf("%06.3f %04d", 2079*6236dae4SAndroid Build Coastguard Worker $timesrvrlog{$testnum} - $timetoolend{$testnum}, $testnum); 2080*6236dae4SAndroid Build Coastguard Worker push @timevrfy, sprintf("%06.3f %04d", 2081*6236dae4SAndroid Build Coastguard Worker $timevrfyend{$testnum} - $timesrvrlog{$testnum}, $testnum); 2082*6236dae4SAndroid Build Coastguard Worker push @timetest, sprintf("%06.3f %04d", 2083*6236dae4SAndroid Build Coastguard Worker $timevrfyend{$testnum} - $timeprepini{$testnum}, $testnum); 2084*6236dae4SAndroid Build Coastguard Worker } 2085*6236dae4SAndroid Build Coastguard Worker } 2086*6236dae4SAndroid Build Coastguard Worker 2087*6236dae4SAndroid Build Coastguard Worker { 2088*6236dae4SAndroid Build Coastguard Worker no warnings 'numeric'; 2089*6236dae4SAndroid Build Coastguard Worker @timesrvr = sort { $b <=> $a } @timesrvr; 2090*6236dae4SAndroid Build Coastguard Worker @timeprep = sort { $b <=> $a } @timeprep; 2091*6236dae4SAndroid Build Coastguard Worker @timetool = sort { $b <=> $a } @timetool; 2092*6236dae4SAndroid Build Coastguard Worker @timelock = sort { $b <=> $a } @timelock; 2093*6236dae4SAndroid Build Coastguard Worker @timevrfy = sort { $b <=> $a } @timevrfy; 2094*6236dae4SAndroid Build Coastguard Worker @timetest = sort { $b <=> $a } @timetest; 2095*6236dae4SAndroid Build Coastguard Worker } 2096*6236dae4SAndroid Build Coastguard Worker 2097*6236dae4SAndroid Build Coastguard Worker logmsg "Spent ". sprintf("%08.3f ", $timesrvrtot) . 2098*6236dae4SAndroid Build Coastguard Worker "seconds starting and verifying test harness servers.\n"; 2099*6236dae4SAndroid Build Coastguard Worker logmsg "Spent ". sprintf("%08.3f ", $timepreptot) . 2100*6236dae4SAndroid Build Coastguard Worker "seconds reading definitions and doing test preparations.\n"; 2101*6236dae4SAndroid Build Coastguard Worker logmsg "Spent ". sprintf("%08.3f ", $timetooltot) . 2102*6236dae4SAndroid Build Coastguard Worker "seconds actually running test tools.\n"; 2103*6236dae4SAndroid Build Coastguard Worker logmsg "Spent ". sprintf("%08.3f ", $timelocktot) . 2104*6236dae4SAndroid Build Coastguard Worker "seconds awaiting server logs lock removal.\n"; 2105*6236dae4SAndroid Build Coastguard Worker logmsg "Spent ". sprintf("%08.3f ", $timevrfytot) . 2106*6236dae4SAndroid Build Coastguard Worker "seconds verifying test results.\n"; 2107*6236dae4SAndroid Build Coastguard Worker logmsg "Spent ". sprintf("%08.3f ", $timetesttot) . 2108*6236dae4SAndroid Build Coastguard Worker "seconds doing all of the above.\n"; 2109*6236dae4SAndroid Build Coastguard Worker 2110*6236dae4SAndroid Build Coastguard Worker $counter = 25; 2111*6236dae4SAndroid Build Coastguard Worker logmsg "\nTest server starting and verification time per test ". 2112*6236dae4SAndroid Build Coastguard Worker sprintf("(%s)...\n\n", (not $fullstats)?"top $counter":"full"); 2113*6236dae4SAndroid Build Coastguard Worker logmsg "-time- test\n"; 2114*6236dae4SAndroid Build Coastguard Worker logmsg "------ ----\n"; 2115*6236dae4SAndroid Build Coastguard Worker foreach my $txt (@timesrvr) { 2116*6236dae4SAndroid Build Coastguard Worker last if((not $fullstats) && (not $counter--)); 2117*6236dae4SAndroid Build Coastguard Worker logmsg "$txt\n"; 2118*6236dae4SAndroid Build Coastguard Worker } 2119*6236dae4SAndroid Build Coastguard Worker 2120*6236dae4SAndroid Build Coastguard Worker $counter = 10; 2121*6236dae4SAndroid Build Coastguard Worker logmsg "\nTest definition reading and preparation time per test ". 2122*6236dae4SAndroid Build Coastguard Worker sprintf("(%s)...\n\n", (not $fullstats)?"top $counter":"full"); 2123*6236dae4SAndroid Build Coastguard Worker logmsg "-time- test\n"; 2124*6236dae4SAndroid Build Coastguard Worker logmsg "------ ----\n"; 2125*6236dae4SAndroid Build Coastguard Worker foreach my $txt (@timeprep) { 2126*6236dae4SAndroid Build Coastguard Worker last if((not $fullstats) && (not $counter--)); 2127*6236dae4SAndroid Build Coastguard Worker logmsg "$txt\n"; 2128*6236dae4SAndroid Build Coastguard Worker } 2129*6236dae4SAndroid Build Coastguard Worker 2130*6236dae4SAndroid Build Coastguard Worker $counter = 25; 2131*6236dae4SAndroid Build Coastguard Worker logmsg "\nTest tool execution time per test ". 2132*6236dae4SAndroid Build Coastguard Worker sprintf("(%s)...\n\n", (not $fullstats)?"top $counter":"full"); 2133*6236dae4SAndroid Build Coastguard Worker logmsg "-time- test\n"; 2134*6236dae4SAndroid Build Coastguard Worker logmsg "------ ----\n"; 2135*6236dae4SAndroid Build Coastguard Worker foreach my $txt (@timetool) { 2136*6236dae4SAndroid Build Coastguard Worker last if((not $fullstats) && (not $counter--)); 2137*6236dae4SAndroid Build Coastguard Worker logmsg "$txt\n"; 2138*6236dae4SAndroid Build Coastguard Worker } 2139*6236dae4SAndroid Build Coastguard Worker 2140*6236dae4SAndroid Build Coastguard Worker $counter = 15; 2141*6236dae4SAndroid Build Coastguard Worker logmsg "\nTest server logs lock removal time per test ". 2142*6236dae4SAndroid Build Coastguard Worker sprintf("(%s)...\n\n", (not $fullstats)?"top $counter":"full"); 2143*6236dae4SAndroid Build Coastguard Worker logmsg "-time- test\n"; 2144*6236dae4SAndroid Build Coastguard Worker logmsg "------ ----\n"; 2145*6236dae4SAndroid Build Coastguard Worker foreach my $txt (@timelock) { 2146*6236dae4SAndroid Build Coastguard Worker last if((not $fullstats) && (not $counter--)); 2147*6236dae4SAndroid Build Coastguard Worker logmsg "$txt\n"; 2148*6236dae4SAndroid Build Coastguard Worker } 2149*6236dae4SAndroid Build Coastguard Worker 2150*6236dae4SAndroid Build Coastguard Worker $counter = 10; 2151*6236dae4SAndroid Build Coastguard Worker logmsg "\nTest results verification time per test ". 2152*6236dae4SAndroid Build Coastguard Worker sprintf("(%s)...\n\n", (not $fullstats)?"top $counter":"full"); 2153*6236dae4SAndroid Build Coastguard Worker logmsg "-time- test\n"; 2154*6236dae4SAndroid Build Coastguard Worker logmsg "------ ----\n"; 2155*6236dae4SAndroid Build Coastguard Worker foreach my $txt (@timevrfy) { 2156*6236dae4SAndroid Build Coastguard Worker last if((not $fullstats) && (not $counter--)); 2157*6236dae4SAndroid Build Coastguard Worker logmsg "$txt\n"; 2158*6236dae4SAndroid Build Coastguard Worker } 2159*6236dae4SAndroid Build Coastguard Worker 2160*6236dae4SAndroid Build Coastguard Worker $counter = 50; 2161*6236dae4SAndroid Build Coastguard Worker logmsg "\nTotal time per test ". 2162*6236dae4SAndroid Build Coastguard Worker sprintf("(%s)...\n\n", (not $fullstats)?"top $counter":"full"); 2163*6236dae4SAndroid Build Coastguard Worker logmsg "-time- test\n"; 2164*6236dae4SAndroid Build Coastguard Worker logmsg "------ ----\n"; 2165*6236dae4SAndroid Build Coastguard Worker foreach my $txt (@timetest) { 2166*6236dae4SAndroid Build Coastguard Worker last if((not $fullstats) && (not $counter--)); 2167*6236dae4SAndroid Build Coastguard Worker logmsg "$txt\n"; 2168*6236dae4SAndroid Build Coastguard Worker } 2169*6236dae4SAndroid Build Coastguard Worker 2170*6236dae4SAndroid Build Coastguard Worker logmsg "\n"; 2171*6236dae4SAndroid Build Coastguard Worker 2172*6236dae4SAndroid Build Coastguard Worker logmsg "::endgroup::\n"; 2173*6236dae4SAndroid Build Coastguard Worker} 2174*6236dae4SAndroid Build Coastguard Worker 2175*6236dae4SAndroid Build Coastguard Worker####################################################################### 2176*6236dae4SAndroid Build Coastguard Worker# returns code indicating why a test was skipped 2177*6236dae4SAndroid Build Coastguard Worker# 0=unknown test, 1=use test result, 2=ignore test result 2178*6236dae4SAndroid Build Coastguard Worker# 2179*6236dae4SAndroid Build Coastguard Workersub ignoreresultcode { 2180*6236dae4SAndroid Build Coastguard Worker my ($testnum)=@_; 2181*6236dae4SAndroid Build Coastguard Worker if(defined $ignoretestcodes{$testnum}) { 2182*6236dae4SAndroid Build Coastguard Worker return $ignoretestcodes{$testnum}; 2183*6236dae4SAndroid Build Coastguard Worker } 2184*6236dae4SAndroid Build Coastguard Worker return 0; 2185*6236dae4SAndroid Build Coastguard Worker} 2186*6236dae4SAndroid Build Coastguard Worker 2187*6236dae4SAndroid Build Coastguard Worker####################################################################### 2188*6236dae4SAndroid Build Coastguard Worker# Put the given runner ID onto the queue of runners ready for a new task 2189*6236dae4SAndroid Build Coastguard Worker# 2190*6236dae4SAndroid Build Coastguard Workersub runnerready { 2191*6236dae4SAndroid Build Coastguard Worker my ($runnerid)=@_; 2192*6236dae4SAndroid Build Coastguard Worker push @runnersidle, $runnerid; 2193*6236dae4SAndroid Build Coastguard Worker} 2194*6236dae4SAndroid Build Coastguard Worker 2195*6236dae4SAndroid Build Coastguard Worker####################################################################### 2196*6236dae4SAndroid Build Coastguard Worker# Create test runners 2197*6236dae4SAndroid Build Coastguard Worker# 2198*6236dae4SAndroid Build Coastguard Workersub createrunners { 2199*6236dae4SAndroid Build Coastguard Worker my ($numrunners)=@_; 2200*6236dae4SAndroid Build Coastguard Worker if(! $numrunners) { 2201*6236dae4SAndroid Build Coastguard Worker $numrunners++; 2202*6236dae4SAndroid Build Coastguard Worker } 2203*6236dae4SAndroid Build Coastguard Worker # create $numrunners runners with minimum 1 2204*6236dae4SAndroid Build Coastguard Worker for my $runnernum (1..$numrunners) { 2205*6236dae4SAndroid Build Coastguard Worker my $dir = getrunnernumlogdir($runnernum); 2206*6236dae4SAndroid Build Coastguard Worker cleardir($dir); 2207*6236dae4SAndroid Build Coastguard Worker mkdir($dir, 0777); 2208*6236dae4SAndroid Build Coastguard Worker $runnerids{$runnernum} = runner_init($dir, $jobs); 2209*6236dae4SAndroid Build Coastguard Worker runnerready($runnerids{$runnernum}); 2210*6236dae4SAndroid Build Coastguard Worker } 2211*6236dae4SAndroid Build Coastguard Worker} 2212*6236dae4SAndroid Build Coastguard Worker 2213*6236dae4SAndroid Build Coastguard Worker####################################################################### 2214*6236dae4SAndroid Build Coastguard Worker# Pick a test runner for the given test 2215*6236dae4SAndroid Build Coastguard Worker# 2216*6236dae4SAndroid Build Coastguard Workersub pickrunner { 2217*6236dae4SAndroid Build Coastguard Worker my ($testnum)=@_; 2218*6236dae4SAndroid Build Coastguard Worker scalar(@runnersidle) || die "No runners available"; 2219*6236dae4SAndroid Build Coastguard Worker 2220*6236dae4SAndroid Build Coastguard Worker return pop @runnersidle; 2221*6236dae4SAndroid Build Coastguard Worker} 2222*6236dae4SAndroid Build Coastguard Worker 2223*6236dae4SAndroid Build Coastguard Worker####################################################################### 2224*6236dae4SAndroid Build Coastguard Worker# Check options to this test program 2225*6236dae4SAndroid Build Coastguard Worker# 2226*6236dae4SAndroid Build Coastguard Worker 2227*6236dae4SAndroid Build Coastguard Worker# Special case for CMake: replace '$TFLAGS' by the contents of the 2228*6236dae4SAndroid Build Coastguard Worker# environment variable (if any). 2229*6236dae4SAndroid Build Coastguard Workerif(@ARGV && $ARGV[-1] eq '$TFLAGS') { 2230*6236dae4SAndroid Build Coastguard Worker pop @ARGV; 2231*6236dae4SAndroid Build Coastguard Worker push(@ARGV, split(' ', $ENV{'TFLAGS'})) if defined($ENV{'TFLAGS'}); 2232*6236dae4SAndroid Build Coastguard Worker} 2233*6236dae4SAndroid Build Coastguard Worker 2234*6236dae4SAndroid Build Coastguard Worker$args = join(' ', @ARGV); 2235*6236dae4SAndroid Build Coastguard Worker 2236*6236dae4SAndroid Build Coastguard Worker$valgrind = checktestcmd("valgrind"); 2237*6236dae4SAndroid Build Coastguard Workermy $number=0; 2238*6236dae4SAndroid Build Coastguard Workermy $fromnum=-1; 2239*6236dae4SAndroid Build Coastguard Workermy @testthis; 2240*6236dae4SAndroid Build Coastguard Workerwhile(@ARGV) { 2241*6236dae4SAndroid Build Coastguard Worker if ($ARGV[0] eq "-v") { 2242*6236dae4SAndroid Build Coastguard Worker # verbose output 2243*6236dae4SAndroid Build Coastguard Worker $verbose=1; 2244*6236dae4SAndroid Build Coastguard Worker } 2245*6236dae4SAndroid Build Coastguard Worker elsif ($ARGV[0] eq "-c") { 2246*6236dae4SAndroid Build Coastguard Worker # use this path to curl instead of default 2247*6236dae4SAndroid Build Coastguard Worker $DBGCURL=$CURL=$ARGV[1]; 2248*6236dae4SAndroid Build Coastguard Worker shift @ARGV; 2249*6236dae4SAndroid Build Coastguard Worker } 2250*6236dae4SAndroid Build Coastguard Worker elsif ($ARGV[0] eq "-vc") { 2251*6236dae4SAndroid Build Coastguard Worker # use this path to a curl used to verify servers 2252*6236dae4SAndroid Build Coastguard Worker 2253*6236dae4SAndroid Build Coastguard Worker # Particularly useful when you introduce a crashing bug somewhere in 2254*6236dae4SAndroid Build Coastguard Worker # the development version as then it won't be able to run any tests 2255*6236dae4SAndroid Build Coastguard Worker # since it can't verify the servers! 2256*6236dae4SAndroid Build Coastguard Worker 2257*6236dae4SAndroid Build Coastguard Worker $VCURL=shell_quote($ARGV[1]); 2258*6236dae4SAndroid Build Coastguard Worker shift @ARGV; 2259*6236dae4SAndroid Build Coastguard Worker } 2260*6236dae4SAndroid Build Coastguard Worker elsif ($ARGV[0] eq "-ac") { 2261*6236dae4SAndroid Build Coastguard Worker # use this curl only to talk to APIs (currently only CI test APIs) 2262*6236dae4SAndroid Build Coastguard Worker $ACURL=shell_quote($ARGV[1]); 2263*6236dae4SAndroid Build Coastguard Worker shift @ARGV; 2264*6236dae4SAndroid Build Coastguard Worker } 2265*6236dae4SAndroid Build Coastguard Worker elsif ($ARGV[0] eq "-bundle") { 2266*6236dae4SAndroid Build Coastguard Worker # use test bundles 2267*6236dae4SAndroid Build Coastguard Worker $bundle=1; 2268*6236dae4SAndroid Build Coastguard Worker } 2269*6236dae4SAndroid Build Coastguard Worker elsif ($ARGV[0] eq "-d") { 2270*6236dae4SAndroid Build Coastguard Worker # have the servers display protocol output 2271*6236dae4SAndroid Build Coastguard Worker $debugprotocol=1; 2272*6236dae4SAndroid Build Coastguard Worker } 2273*6236dae4SAndroid Build Coastguard Worker elsif($ARGV[0] eq "-e") { 2274*6236dae4SAndroid Build Coastguard Worker # run the tests cases event based if possible 2275*6236dae4SAndroid Build Coastguard Worker $run_event_based=1; 2276*6236dae4SAndroid Build Coastguard Worker } 2277*6236dae4SAndroid Build Coastguard Worker elsif($ARGV[0] eq "-f") { 2278*6236dae4SAndroid Build Coastguard Worker # force - run the test case even if listed in DISABLED 2279*6236dae4SAndroid Build Coastguard Worker $run_disabled=1; 2280*6236dae4SAndroid Build Coastguard Worker } 2281*6236dae4SAndroid Build Coastguard Worker elsif($ARGV[0] eq "-E") { 2282*6236dae4SAndroid Build Coastguard Worker # load additional reasons to skip tests 2283*6236dae4SAndroid Build Coastguard Worker shift @ARGV; 2284*6236dae4SAndroid Build Coastguard Worker my $exclude_file = $ARGV[0]; 2285*6236dae4SAndroid Build Coastguard Worker open(my $fd, "<", $exclude_file) or die "Couldn't open '$exclude_file': $!"; 2286*6236dae4SAndroid Build Coastguard Worker while(my $line = <$fd>) { 2287*6236dae4SAndroid Build Coastguard Worker next if ($line =~ /^#/); 2288*6236dae4SAndroid Build Coastguard Worker chomp $line; 2289*6236dae4SAndroid Build Coastguard Worker my ($type, $patterns, $skip_reason) = split(/\s*:\s*/, $line, 3); 2290*6236dae4SAndroid Build Coastguard Worker 2291*6236dae4SAndroid Build Coastguard Worker die "Unsupported type: $type\n" if($type !~ /^keyword|test|tool$/); 2292*6236dae4SAndroid Build Coastguard Worker 2293*6236dae4SAndroid Build Coastguard Worker foreach my $pattern (split(/,/, $patterns)) { 2294*6236dae4SAndroid Build Coastguard Worker if($type eq "test") { 2295*6236dae4SAndroid Build Coastguard Worker # Strip leading zeros in the test number 2296*6236dae4SAndroid Build Coastguard Worker $pattern = int($pattern); 2297*6236dae4SAndroid Build Coastguard Worker } 2298*6236dae4SAndroid Build Coastguard Worker $custom_skip_reasons{$type}{$pattern} = $skip_reason; 2299*6236dae4SAndroid Build Coastguard Worker } 2300*6236dae4SAndroid Build Coastguard Worker } 2301*6236dae4SAndroid Build Coastguard Worker close($fd); 2302*6236dae4SAndroid Build Coastguard Worker } 2303*6236dae4SAndroid Build Coastguard Worker elsif ($ARGV[0] eq "-g") { 2304*6236dae4SAndroid Build Coastguard Worker # run this test with gdb 2305*6236dae4SAndroid Build Coastguard Worker $gdbthis=1; 2306*6236dae4SAndroid Build Coastguard Worker } 2307*6236dae4SAndroid Build Coastguard Worker elsif ($ARGV[0] eq "-gl") { 2308*6236dae4SAndroid Build Coastguard Worker # run this test with lldb 2309*6236dae4SAndroid Build Coastguard Worker $gdbthis=2; 2310*6236dae4SAndroid Build Coastguard Worker } 2311*6236dae4SAndroid Build Coastguard Worker elsif ($ARGV[0] eq "-gw") { 2312*6236dae4SAndroid Build Coastguard Worker # run this test with windowed gdb 2313*6236dae4SAndroid Build Coastguard Worker $gdbthis=1; 2314*6236dae4SAndroid Build Coastguard Worker $gdbxwin=1; 2315*6236dae4SAndroid Build Coastguard Worker } 2316*6236dae4SAndroid Build Coastguard Worker elsif($ARGV[0] eq "-s") { 2317*6236dae4SAndroid Build Coastguard Worker # short output 2318*6236dae4SAndroid Build Coastguard Worker $short=1; 2319*6236dae4SAndroid Build Coastguard Worker } 2320*6236dae4SAndroid Build Coastguard Worker elsif($ARGV[0] eq "-am") { 2321*6236dae4SAndroid Build Coastguard Worker # automake-style output 2322*6236dae4SAndroid Build Coastguard Worker $short=1; 2323*6236dae4SAndroid Build Coastguard Worker $automakestyle=1; 2324*6236dae4SAndroid Build Coastguard Worker } 2325*6236dae4SAndroid Build Coastguard Worker elsif($ARGV[0] eq "-n") { 2326*6236dae4SAndroid Build Coastguard Worker # no valgrind 2327*6236dae4SAndroid Build Coastguard Worker undef $valgrind; 2328*6236dae4SAndroid Build Coastguard Worker } 2329*6236dae4SAndroid Build Coastguard Worker elsif($ARGV[0] eq "--no-debuginfod") { 2330*6236dae4SAndroid Build Coastguard Worker # disable the valgrind debuginfod functionality 2331*6236dae4SAndroid Build Coastguard Worker $no_debuginfod = 1; 2332*6236dae4SAndroid Build Coastguard Worker } 2333*6236dae4SAndroid Build Coastguard Worker elsif ($ARGV[0] eq "-R") { 2334*6236dae4SAndroid Build Coastguard Worker # execute in scrambled order 2335*6236dae4SAndroid Build Coastguard Worker $scrambleorder=1; 2336*6236dae4SAndroid Build Coastguard Worker } 2337*6236dae4SAndroid Build Coastguard Worker elsif($ARGV[0] =~ /^-t(.*)/) { 2338*6236dae4SAndroid Build Coastguard Worker # torture 2339*6236dae4SAndroid Build Coastguard Worker $torture=1; 2340*6236dae4SAndroid Build Coastguard Worker my $xtra = $1; 2341*6236dae4SAndroid Build Coastguard Worker 2342*6236dae4SAndroid Build Coastguard Worker if($xtra =~ s/(\d+)$//) { 2343*6236dae4SAndroid Build Coastguard Worker $tortalloc = $1; 2344*6236dae4SAndroid Build Coastguard Worker } 2345*6236dae4SAndroid Build Coastguard Worker } 2346*6236dae4SAndroid Build Coastguard Worker elsif($ARGV[0] =~ /--shallow=(\d+)/) { 2347*6236dae4SAndroid Build Coastguard Worker # Fail no more than this amount per tests when running 2348*6236dae4SAndroid Build Coastguard Worker # torture. 2349*6236dae4SAndroid Build Coastguard Worker my ($num)=($1); 2350*6236dae4SAndroid Build Coastguard Worker $shallow=$num; 2351*6236dae4SAndroid Build Coastguard Worker } 2352*6236dae4SAndroid Build Coastguard Worker elsif($ARGV[0] =~ /--repeat=(\d+)/) { 2353*6236dae4SAndroid Build Coastguard Worker # Repeat-run the given tests this many times 2354*6236dae4SAndroid Build Coastguard Worker $repeat = $1; 2355*6236dae4SAndroid Build Coastguard Worker } 2356*6236dae4SAndroid Build Coastguard Worker elsif($ARGV[0] =~ /--seed=(\d+)/) { 2357*6236dae4SAndroid Build Coastguard Worker # Set a fixed random seed (used for -R and --shallow) 2358*6236dae4SAndroid Build Coastguard Worker $randseed = $1; 2359*6236dae4SAndroid Build Coastguard Worker } 2360*6236dae4SAndroid Build Coastguard Worker elsif($ARGV[0] eq "-a") { 2361*6236dae4SAndroid Build Coastguard Worker # continue anyway, even if a test fail 2362*6236dae4SAndroid Build Coastguard Worker $anyway=1; 2363*6236dae4SAndroid Build Coastguard Worker } 2364*6236dae4SAndroid Build Coastguard Worker elsif($ARGV[0] eq "-o") { 2365*6236dae4SAndroid Build Coastguard Worker shift @ARGV; 2366*6236dae4SAndroid Build Coastguard Worker if ($ARGV[0] =~ /^(\w+)=([\w.:\/\[\]-]+)$/) { 2367*6236dae4SAndroid Build Coastguard Worker my ($variable, $value) = ($1, $2); 2368*6236dae4SAndroid Build Coastguard Worker eval "\$$variable='$value'" or die "Failed to set \$$variable to $value: $@"; 2369*6236dae4SAndroid Build Coastguard Worker } else { 2370*6236dae4SAndroid Build Coastguard Worker die "Failed to parse '-o $ARGV[0]'. May contain unexpected characters.\n"; 2371*6236dae4SAndroid Build Coastguard Worker } 2372*6236dae4SAndroid Build Coastguard Worker } 2373*6236dae4SAndroid Build Coastguard Worker elsif($ARGV[0] eq "-p") { 2374*6236dae4SAndroid Build Coastguard Worker $postmortem=1; 2375*6236dae4SAndroid Build Coastguard Worker } 2376*6236dae4SAndroid Build Coastguard Worker elsif($ARGV[0] eq "-P") { 2377*6236dae4SAndroid Build Coastguard Worker shift @ARGV; 2378*6236dae4SAndroid Build Coastguard Worker $proxy_address=$ARGV[0]; 2379*6236dae4SAndroid Build Coastguard Worker } 2380*6236dae4SAndroid Build Coastguard Worker elsif($ARGV[0] eq "-L") { 2381*6236dae4SAndroid Build Coastguard Worker # require additional library file 2382*6236dae4SAndroid Build Coastguard Worker shift @ARGV; 2383*6236dae4SAndroid Build Coastguard Worker require $ARGV[0]; 2384*6236dae4SAndroid Build Coastguard Worker } 2385*6236dae4SAndroid Build Coastguard Worker elsif($ARGV[0] eq "-l") { 2386*6236dae4SAndroid Build Coastguard Worker # lists the test case names only 2387*6236dae4SAndroid Build Coastguard Worker $listonly=1; 2388*6236dae4SAndroid Build Coastguard Worker } 2389*6236dae4SAndroid Build Coastguard Worker elsif($ARGV[0] =~ /^-j(.*)/) { 2390*6236dae4SAndroid Build Coastguard Worker # parallel jobs 2391*6236dae4SAndroid Build Coastguard Worker $jobs=1; 2392*6236dae4SAndroid Build Coastguard Worker my $xtra = $1; 2393*6236dae4SAndroid Build Coastguard Worker if($xtra =~ s/(\d+)$//) { 2394*6236dae4SAndroid Build Coastguard Worker $jobs = $1; 2395*6236dae4SAndroid Build Coastguard Worker } 2396*6236dae4SAndroid Build Coastguard Worker } 2397*6236dae4SAndroid Build Coastguard Worker elsif($ARGV[0] eq "-k") { 2398*6236dae4SAndroid Build Coastguard Worker # keep stdout and stderr files after tests 2399*6236dae4SAndroid Build Coastguard Worker $keepoutfiles=1; 2400*6236dae4SAndroid Build Coastguard Worker } 2401*6236dae4SAndroid Build Coastguard Worker elsif($ARGV[0] eq "-r") { 2402*6236dae4SAndroid Build Coastguard Worker # run time statistics needs Time::HiRes 2403*6236dae4SAndroid Build Coastguard Worker if($Time::HiRes::VERSION) { 2404*6236dae4SAndroid Build Coastguard Worker # presize hashes appropriately to hold an entire test run 2405*6236dae4SAndroid Build Coastguard Worker keys(%timeprepini) = 2000; 2406*6236dae4SAndroid Build Coastguard Worker keys(%timesrvrini) = 2000; 2407*6236dae4SAndroid Build Coastguard Worker keys(%timesrvrend) = 2000; 2408*6236dae4SAndroid Build Coastguard Worker keys(%timetoolini) = 2000; 2409*6236dae4SAndroid Build Coastguard Worker keys(%timetoolend) = 2000; 2410*6236dae4SAndroid Build Coastguard Worker keys(%timesrvrlog) = 2000; 2411*6236dae4SAndroid Build Coastguard Worker keys(%timevrfyend) = 2000; 2412*6236dae4SAndroid Build Coastguard Worker $timestats=1; 2413*6236dae4SAndroid Build Coastguard Worker $fullstats=0; 2414*6236dae4SAndroid Build Coastguard Worker } 2415*6236dae4SAndroid Build Coastguard Worker } 2416*6236dae4SAndroid Build Coastguard Worker elsif($ARGV[0] eq "-rf") { 2417*6236dae4SAndroid Build Coastguard Worker # run time statistics needs Time::HiRes 2418*6236dae4SAndroid Build Coastguard Worker if($Time::HiRes::VERSION) { 2419*6236dae4SAndroid Build Coastguard Worker # presize hashes appropriately to hold an entire test run 2420*6236dae4SAndroid Build Coastguard Worker keys(%timeprepini) = 2000; 2421*6236dae4SAndroid Build Coastguard Worker keys(%timesrvrini) = 2000; 2422*6236dae4SAndroid Build Coastguard Worker keys(%timesrvrend) = 2000; 2423*6236dae4SAndroid Build Coastguard Worker keys(%timetoolini) = 2000; 2424*6236dae4SAndroid Build Coastguard Worker keys(%timetoolend) = 2000; 2425*6236dae4SAndroid Build Coastguard Worker keys(%timesrvrlog) = 2000; 2426*6236dae4SAndroid Build Coastguard Worker keys(%timevrfyend) = 2000; 2427*6236dae4SAndroid Build Coastguard Worker $timestats=1; 2428*6236dae4SAndroid Build Coastguard Worker $fullstats=1; 2429*6236dae4SAndroid Build Coastguard Worker } 2430*6236dae4SAndroid Build Coastguard Worker } 2431*6236dae4SAndroid Build Coastguard Worker elsif($ARGV[0] eq "-rm") { 2432*6236dae4SAndroid Build Coastguard Worker # force removal of files by killing locking processes 2433*6236dae4SAndroid Build Coastguard Worker $clearlocks=1; 2434*6236dae4SAndroid Build Coastguard Worker } 2435*6236dae4SAndroid Build Coastguard Worker elsif($ARGV[0] eq "-u") { 2436*6236dae4SAndroid Build Coastguard Worker # error instead of warning on server unexpectedly alive 2437*6236dae4SAndroid Build Coastguard Worker $err_unexpected=1; 2438*6236dae4SAndroid Build Coastguard Worker } 2439*6236dae4SAndroid Build Coastguard Worker elsif(($ARGV[0] eq "-h") || ($ARGV[0] eq "--help")) { 2440*6236dae4SAndroid Build Coastguard Worker # show help text 2441*6236dae4SAndroid Build Coastguard Worker print <<"EOHELP" 2442*6236dae4SAndroid Build Coastguard WorkerUsage: runtests.pl [options] [test selection(s)] 2443*6236dae4SAndroid Build Coastguard Worker -a continue even if a test fails 2444*6236dae4SAndroid Build Coastguard Worker -ac path use this curl only to talk to APIs (currently only CI test APIs) 2445*6236dae4SAndroid Build Coastguard Worker -am automake style output PASS/FAIL: [number] [name] 2446*6236dae4SAndroid Build Coastguard Worker -bundle use test bundles 2447*6236dae4SAndroid Build Coastguard Worker -c path use this curl executable 2448*6236dae4SAndroid Build Coastguard Worker -d display server debug info 2449*6236dae4SAndroid Build Coastguard Worker -e event-based execution 2450*6236dae4SAndroid Build Coastguard Worker -E file load the specified file to exclude certain tests 2451*6236dae4SAndroid Build Coastguard Worker -f forcibly run even if disabled 2452*6236dae4SAndroid Build Coastguard Worker -g run the test case with gdb 2453*6236dae4SAndroid Build Coastguard Worker -gw run the test case with gdb as a windowed application 2454*6236dae4SAndroid Build Coastguard Worker -h this help text 2455*6236dae4SAndroid Build Coastguard Worker -j[N] spawn this number of processes to run tests (default 0) 2456*6236dae4SAndroid Build Coastguard Worker -k keep stdout and stderr files present after tests 2457*6236dae4SAndroid Build Coastguard Worker -L path require an additional perl library file to replace certain functions 2458*6236dae4SAndroid Build Coastguard Worker -l list all test case names/descriptions 2459*6236dae4SAndroid Build Coastguard Worker -n no valgrind 2460*6236dae4SAndroid Build Coastguard Worker --no-debuginfod disable the valgrind debuginfod functionality 2461*6236dae4SAndroid Build Coastguard Worker -o variable=value set internal variable to the specified value 2462*6236dae4SAndroid Build Coastguard Worker -P proxy use the specified proxy 2463*6236dae4SAndroid Build Coastguard Worker -p print log file contents when a test fails 2464*6236dae4SAndroid Build Coastguard Worker -R scrambled order (uses the random seed, see --seed) 2465*6236dae4SAndroid Build Coastguard Worker -r run time statistics 2466*6236dae4SAndroid Build Coastguard Worker -rf full run time statistics 2467*6236dae4SAndroid Build Coastguard Worker -rm force removal of files by killing locking processes (Windows only) 2468*6236dae4SAndroid Build Coastguard Worker --repeat=[num] run the given tests this many times 2469*6236dae4SAndroid Build Coastguard Worker -s short output 2470*6236dae4SAndroid Build Coastguard Worker --seed=[num] set the random seed to a fixed number 2471*6236dae4SAndroid Build Coastguard Worker --shallow=[num] randomly makes the torture tests "thinner" 2472*6236dae4SAndroid Build Coastguard Worker -t[N] torture (simulate function failures); N means fail Nth function 2473*6236dae4SAndroid Build Coastguard Worker -u error instead of warning on server unexpectedly alive 2474*6236dae4SAndroid Build Coastguard Worker -v verbose output 2475*6236dae4SAndroid Build Coastguard Worker -vc path use this curl only to verify the existing servers 2476*6236dae4SAndroid Build Coastguard Worker [num] like "5 6 9" or " 5 to 22 " to run those tests only 2477*6236dae4SAndroid Build Coastguard Worker [!num] like "!5 !6 !9" to disable those tests 2478*6236dae4SAndroid Build Coastguard Worker [~num] like "~5 ~6 ~9" to ignore the result of those tests 2479*6236dae4SAndroid Build Coastguard Worker [keyword] like "IPv6" to select only tests containing the key word 2480*6236dae4SAndroid Build Coastguard Worker [!keyword] like "!cookies" to disable any tests containing the key word 2481*6236dae4SAndroid Build Coastguard Worker [~keyword] like "~cookies" to ignore results of tests containing key word 2482*6236dae4SAndroid Build Coastguard WorkerEOHELP 2483*6236dae4SAndroid Build Coastguard Worker ; 2484*6236dae4SAndroid Build Coastguard Worker exit; 2485*6236dae4SAndroid Build Coastguard Worker } 2486*6236dae4SAndroid Build Coastguard Worker elsif($ARGV[0] =~ /^(\d+)/) { 2487*6236dae4SAndroid Build Coastguard Worker $number = $1; 2488*6236dae4SAndroid Build Coastguard Worker if($fromnum >= 0) { 2489*6236dae4SAndroid Build Coastguard Worker for my $n ($fromnum .. $number) { 2490*6236dae4SAndroid Build Coastguard Worker push @testthis, $n; 2491*6236dae4SAndroid Build Coastguard Worker } 2492*6236dae4SAndroid Build Coastguard Worker $fromnum = -1; 2493*6236dae4SAndroid Build Coastguard Worker } 2494*6236dae4SAndroid Build Coastguard Worker else { 2495*6236dae4SAndroid Build Coastguard Worker push @testthis, $1; 2496*6236dae4SAndroid Build Coastguard Worker } 2497*6236dae4SAndroid Build Coastguard Worker } 2498*6236dae4SAndroid Build Coastguard Worker elsif($ARGV[0] =~ /^to$/i) { 2499*6236dae4SAndroid Build Coastguard Worker $fromnum = $number+1; 2500*6236dae4SAndroid Build Coastguard Worker } 2501*6236dae4SAndroid Build Coastguard Worker elsif($ARGV[0] =~ /^!(\d+)/) { 2502*6236dae4SAndroid Build Coastguard Worker $fromnum = -1; 2503*6236dae4SAndroid Build Coastguard Worker $disabled{$1}=$1; 2504*6236dae4SAndroid Build Coastguard Worker } 2505*6236dae4SAndroid Build Coastguard Worker elsif($ARGV[0] =~ /^~(\d+)/) { 2506*6236dae4SAndroid Build Coastguard Worker $fromnum = -1; 2507*6236dae4SAndroid Build Coastguard Worker $ignored{$1}=$1; 2508*6236dae4SAndroid Build Coastguard Worker } 2509*6236dae4SAndroid Build Coastguard Worker elsif($ARGV[0] =~ /^!(.+)/) { 2510*6236dae4SAndroid Build Coastguard Worker $disabled_keywords{lc($1)}=$1; 2511*6236dae4SAndroid Build Coastguard Worker } 2512*6236dae4SAndroid Build Coastguard Worker elsif($ARGV[0] =~ /^~(.+)/) { 2513*6236dae4SAndroid Build Coastguard Worker $ignored_keywords{lc($1)}=$1; 2514*6236dae4SAndroid Build Coastguard Worker } 2515*6236dae4SAndroid Build Coastguard Worker elsif($ARGV[0] =~ /^([-[{a-zA-Z].*)/) { 2516*6236dae4SAndroid Build Coastguard Worker $enabled_keywords{lc($1)}=$1; 2517*6236dae4SAndroid Build Coastguard Worker } 2518*6236dae4SAndroid Build Coastguard Worker else { 2519*6236dae4SAndroid Build Coastguard Worker print "Unknown option: $ARGV[0]\n"; 2520*6236dae4SAndroid Build Coastguard Worker exit; 2521*6236dae4SAndroid Build Coastguard Worker } 2522*6236dae4SAndroid Build Coastguard Worker shift @ARGV; 2523*6236dae4SAndroid Build Coastguard Worker} 2524*6236dae4SAndroid Build Coastguard Worker 2525*6236dae4SAndroid Build Coastguard Workerdelete $ENV{'DEBUGINFOD_URLS'} if($ENV{'DEBUGINFOD_URLS'} && $no_debuginfod); 2526*6236dae4SAndroid Build Coastguard Worker 2527*6236dae4SAndroid Build Coastguard Workerif(!$randseed) { 2528*6236dae4SAndroid Build Coastguard Worker my ($sec,$min,$hour,$mday,$mon,$year,$wday,$yday,$isdst) = 2529*6236dae4SAndroid Build Coastguard Worker localtime(time); 2530*6236dae4SAndroid Build Coastguard Worker # seed of the month. December 2019 becomes 201912 2531*6236dae4SAndroid Build Coastguard Worker $randseed = ($year+1900)*100 + $mon+1; 2532*6236dae4SAndroid Build Coastguard Worker print "Using curl: $CURL\n"; 2533*6236dae4SAndroid Build Coastguard Worker open(my $curlvh, "-|", shell_quote($CURL) . " --version 2>$dev_null") || 2534*6236dae4SAndroid Build Coastguard Worker die "could not get curl version!"; 2535*6236dae4SAndroid Build Coastguard Worker my @c = <$curlvh>; 2536*6236dae4SAndroid Build Coastguard Worker close($curlvh) || die "could not get curl version!"; 2537*6236dae4SAndroid Build Coastguard Worker # use the first line of output and get the md5 out of it 2538*6236dae4SAndroid Build Coastguard Worker my $str = md5($c[0]); 2539*6236dae4SAndroid Build Coastguard Worker $randseed += unpack('S', $str); # unsigned 16 bit value 2540*6236dae4SAndroid Build Coastguard Worker} 2541*6236dae4SAndroid Build Coastguard Workersrand $randseed; 2542*6236dae4SAndroid Build Coastguard Worker 2543*6236dae4SAndroid Build Coastguard Workerif(@testthis && ($testthis[0] ne "")) { 2544*6236dae4SAndroid Build Coastguard Worker $TESTCASES=join(" ", @testthis); 2545*6236dae4SAndroid Build Coastguard Worker} 2546*6236dae4SAndroid Build Coastguard Worker 2547*6236dae4SAndroid Build Coastguard Workerif($valgrind) { 2548*6236dae4SAndroid Build Coastguard Worker # we have found valgrind on the host, use it 2549*6236dae4SAndroid Build Coastguard Worker 2550*6236dae4SAndroid Build Coastguard Worker # verify that we can invoke it fine 2551*6236dae4SAndroid Build Coastguard Worker my $code = runclient("valgrind >$dev_null 2>&1"); 2552*6236dae4SAndroid Build Coastguard Worker 2553*6236dae4SAndroid Build Coastguard Worker if(($code>>8) != 1) { 2554*6236dae4SAndroid Build Coastguard Worker #logmsg "Valgrind failure, disable it\n"; 2555*6236dae4SAndroid Build Coastguard Worker undef $valgrind; 2556*6236dae4SAndroid Build Coastguard Worker } else { 2557*6236dae4SAndroid Build Coastguard Worker 2558*6236dae4SAndroid Build Coastguard Worker # since valgrind 2.1.x, '--tool' option is mandatory 2559*6236dae4SAndroid Build Coastguard Worker # use it, if it is supported by the version installed on the system 2560*6236dae4SAndroid Build Coastguard Worker # (this happened in 2003, so we could probably don't need to care about 2561*6236dae4SAndroid Build Coastguard Worker # that old version any longer and just delete this check) 2562*6236dae4SAndroid Build Coastguard Worker runclient("valgrind --help 2>&1 | grep -- --tool >$dev_null 2>&1"); 2563*6236dae4SAndroid Build Coastguard Worker if (($? >> 8)) { 2564*6236dae4SAndroid Build Coastguard Worker $valgrind_tool=""; 2565*6236dae4SAndroid Build Coastguard Worker } 2566*6236dae4SAndroid Build Coastguard Worker open(my $curlh, "<", "$CURL"); 2567*6236dae4SAndroid Build Coastguard Worker my $l = <$curlh>; 2568*6236dae4SAndroid Build Coastguard Worker if($l =~ /^\#\!/) { 2569*6236dae4SAndroid Build Coastguard Worker # A shell script. This is typically when built with libtool, 2570*6236dae4SAndroid Build Coastguard Worker $valgrind="../libtool --mode=execute $valgrind"; 2571*6236dae4SAndroid Build Coastguard Worker } 2572*6236dae4SAndroid Build Coastguard Worker close($curlh); 2573*6236dae4SAndroid Build Coastguard Worker 2574*6236dae4SAndroid Build Coastguard Worker # valgrind 3 renamed the --logfile option to --log-file!!! 2575*6236dae4SAndroid Build Coastguard Worker # (this happened in 2005, so we could probably don't need to care about 2576*6236dae4SAndroid Build Coastguard Worker # that old version any longer and just delete this check) 2577*6236dae4SAndroid Build Coastguard Worker my $ver=join(' ', runclientoutput("valgrind --version")); 2578*6236dae4SAndroid Build Coastguard Worker # cut off all but digits and dots 2579*6236dae4SAndroid Build Coastguard Worker $ver =~ s/[^0-9.]//g; 2580*6236dae4SAndroid Build Coastguard Worker 2581*6236dae4SAndroid Build Coastguard Worker if($ver =~ /^(\d+)/) { 2582*6236dae4SAndroid Build Coastguard Worker $ver = $1; 2583*6236dae4SAndroid Build Coastguard Worker if($ver < 3) { 2584*6236dae4SAndroid Build Coastguard Worker $valgrind_logfile="--logfile"; 2585*6236dae4SAndroid Build Coastguard Worker } 2586*6236dae4SAndroid Build Coastguard Worker } 2587*6236dae4SAndroid Build Coastguard Worker } 2588*6236dae4SAndroid Build Coastguard Worker} 2589*6236dae4SAndroid Build Coastguard Worker 2590*6236dae4SAndroid Build Coastguard Workerif ($gdbthis) { 2591*6236dae4SAndroid Build Coastguard Worker # open the executable curl and read the first 4 bytes of it 2592*6236dae4SAndroid Build Coastguard Worker open(my $check, "<", "$CURL"); 2593*6236dae4SAndroid Build Coastguard Worker my $c; 2594*6236dae4SAndroid Build Coastguard Worker sysread $check, $c, 4; 2595*6236dae4SAndroid Build Coastguard Worker close($check); 2596*6236dae4SAndroid Build Coastguard Worker if($c eq "#! /") { 2597*6236dae4SAndroid Build Coastguard Worker # A shell script. This is typically when built with libtool, 2598*6236dae4SAndroid Build Coastguard Worker $libtool = 1; 2599*6236dae4SAndroid Build Coastguard Worker $gdb = "../libtool --mode=execute gdb"; 2600*6236dae4SAndroid Build Coastguard Worker } 2601*6236dae4SAndroid Build Coastguard Worker} 2602*6236dae4SAndroid Build Coastguard Worker 2603*6236dae4SAndroid Build Coastguard Worker####################################################################### 2604*6236dae4SAndroid Build Coastguard Worker# clear and create logging directory: 2605*6236dae4SAndroid Build Coastguard Worker# 2606*6236dae4SAndroid Build Coastguard Worker 2607*6236dae4SAndroid Build Coastguard Worker# TODO: figure how to get around this. This dir is needed for checksystemfeatures() 2608*6236dae4SAndroid Build Coastguard Worker# Maybe create & use & delete a temporary directory in that function 2609*6236dae4SAndroid Build Coastguard Workercleardir($LOGDIR); 2610*6236dae4SAndroid Build Coastguard Workermkdir($LOGDIR, 0777); 2611*6236dae4SAndroid Build Coastguard Workermkdir("$LOGDIR/$LOCKDIR", 0777); 2612*6236dae4SAndroid Build Coastguard Worker 2613*6236dae4SAndroid Build Coastguard Worker####################################################################### 2614*6236dae4SAndroid Build Coastguard Worker# initialize some variables 2615*6236dae4SAndroid Build Coastguard Worker# 2616*6236dae4SAndroid Build Coastguard Worker 2617*6236dae4SAndroid Build Coastguard Workerget_disttests(); 2618*6236dae4SAndroid Build Coastguard Workerif(!$jobs) { 2619*6236dae4SAndroid Build Coastguard Worker # Disable buffered logging with only one test job 2620*6236dae4SAndroid Build Coastguard Worker setlogfunc(\&logmsg); 2621*6236dae4SAndroid Build Coastguard Worker} 2622*6236dae4SAndroid Build Coastguard Worker 2623*6236dae4SAndroid Build Coastguard Worker####################################################################### 2624*6236dae4SAndroid Build Coastguard Worker# Output curl version and host info being tested 2625*6236dae4SAndroid Build Coastguard Worker# 2626*6236dae4SAndroid Build Coastguard Worker 2627*6236dae4SAndroid Build Coastguard Workerif(!$listonly) { 2628*6236dae4SAndroid Build Coastguard Worker checksystemfeatures(); 2629*6236dae4SAndroid Build Coastguard Worker} 2630*6236dae4SAndroid Build Coastguard Worker 2631*6236dae4SAndroid Build Coastguard Worker####################################################################### 2632*6236dae4SAndroid Build Coastguard Worker# Output information about the curl build 2633*6236dae4SAndroid Build Coastguard Worker# 2634*6236dae4SAndroid Build Coastguard Workerif(!$listonly) { 2635*6236dae4SAndroid Build Coastguard Worker if(open(my $fd, "<", "../buildinfo.txt")) { 2636*6236dae4SAndroid Build Coastguard Worker while(my $line = <$fd>) { 2637*6236dae4SAndroid Build Coastguard Worker chomp $line; 2638*6236dae4SAndroid Build Coastguard Worker if($line && $line !~ /^#/) { 2639*6236dae4SAndroid Build Coastguard Worker logmsg("* $line\n"); 2640*6236dae4SAndroid Build Coastguard Worker } 2641*6236dae4SAndroid Build Coastguard Worker } 2642*6236dae4SAndroid Build Coastguard Worker close($fd); 2643*6236dae4SAndroid Build Coastguard Worker } 2644*6236dae4SAndroid Build Coastguard Worker} 2645*6236dae4SAndroid Build Coastguard Worker 2646*6236dae4SAndroid Build Coastguard Worker####################################################################### 2647*6236dae4SAndroid Build Coastguard Worker# initialize configuration needed to set up servers 2648*6236dae4SAndroid Build Coastguard Worker# TODO: rearrange things so this can be called only in runner_init() 2649*6236dae4SAndroid Build Coastguard Worker# 2650*6236dae4SAndroid Build Coastguard Workerinitserverconfig(); 2651*6236dae4SAndroid Build Coastguard Worker 2652*6236dae4SAndroid Build Coastguard Workerif(!$listonly) { 2653*6236dae4SAndroid Build Coastguard Worker # these can only be displayed after initserverconfig() has been called 2654*6236dae4SAndroid Build Coastguard Worker displayserverfeatures(); 2655*6236dae4SAndroid Build Coastguard Worker 2656*6236dae4SAndroid Build Coastguard Worker # globally disabled tests 2657*6236dae4SAndroid Build Coastguard Worker disabledtests("$TESTDIR/DISABLED"); 2658*6236dae4SAndroid Build Coastguard Worker} 2659*6236dae4SAndroid Build Coastguard Worker 2660*6236dae4SAndroid Build Coastguard Worker####################################################################### 2661*6236dae4SAndroid Build Coastguard Worker# Fetch all disabled tests, if there are any 2662*6236dae4SAndroid Build Coastguard Worker# 2663*6236dae4SAndroid Build Coastguard Worker 2664*6236dae4SAndroid Build Coastguard Workersub disabledtests { 2665*6236dae4SAndroid Build Coastguard Worker my ($file) = @_; 2666*6236dae4SAndroid Build Coastguard Worker my @input; 2667*6236dae4SAndroid Build Coastguard Worker 2668*6236dae4SAndroid Build Coastguard Worker if(open(my $disabledh, "<", "$file")) { 2669*6236dae4SAndroid Build Coastguard Worker while(<$disabledh>) { 2670*6236dae4SAndroid Build Coastguard Worker if(/^ *\#/) { 2671*6236dae4SAndroid Build Coastguard Worker # allow comments 2672*6236dae4SAndroid Build Coastguard Worker next; 2673*6236dae4SAndroid Build Coastguard Worker } 2674*6236dae4SAndroid Build Coastguard Worker push @input, $_; 2675*6236dae4SAndroid Build Coastguard Worker } 2676*6236dae4SAndroid Build Coastguard Worker close($disabledh); 2677*6236dae4SAndroid Build Coastguard Worker 2678*6236dae4SAndroid Build Coastguard Worker # preprocess the input to make conditionally disabled tests depending 2679*6236dae4SAndroid Build Coastguard Worker # on variables 2680*6236dae4SAndroid Build Coastguard Worker my @pp = prepro(0, @input); 2681*6236dae4SAndroid Build Coastguard Worker for my $t (@pp) { 2682*6236dae4SAndroid Build Coastguard Worker if($t =~ /(\d+)/) { 2683*6236dae4SAndroid Build Coastguard Worker my ($n) = $1; 2684*6236dae4SAndroid Build Coastguard Worker $disabled{$n}=$n; # disable this test number 2685*6236dae4SAndroid Build Coastguard Worker if(! -f "$srcdir/data/test$n") { 2686*6236dae4SAndroid Build Coastguard Worker print STDERR "WARNING! Non-existing test $n in $file!\n"; 2687*6236dae4SAndroid Build Coastguard Worker # fail hard to make user notice 2688*6236dae4SAndroid Build Coastguard Worker exit 1; 2689*6236dae4SAndroid Build Coastguard Worker } 2690*6236dae4SAndroid Build Coastguard Worker logmsg "DISABLED: test $n\n" if ($verbose); 2691*6236dae4SAndroid Build Coastguard Worker } 2692*6236dae4SAndroid Build Coastguard Worker else { 2693*6236dae4SAndroid Build Coastguard Worker print STDERR "$file: rubbish content: $t\n"; 2694*6236dae4SAndroid Build Coastguard Worker exit 2; 2695*6236dae4SAndroid Build Coastguard Worker } 2696*6236dae4SAndroid Build Coastguard Worker } 2697*6236dae4SAndroid Build Coastguard Worker } 2698*6236dae4SAndroid Build Coastguard Worker else { 2699*6236dae4SAndroid Build Coastguard Worker print STDERR "Cannot open $file, exiting\n"; 2700*6236dae4SAndroid Build Coastguard Worker exit 3; 2701*6236dae4SAndroid Build Coastguard Worker } 2702*6236dae4SAndroid Build Coastguard Worker} 2703*6236dae4SAndroid Build Coastguard Worker 2704*6236dae4SAndroid Build Coastguard Worker####################################################################### 2705*6236dae4SAndroid Build Coastguard Worker# If 'all' tests are requested, find out all test numbers 2706*6236dae4SAndroid Build Coastguard Worker# 2707*6236dae4SAndroid Build Coastguard Worker 2708*6236dae4SAndroid Build Coastguard Workerif ( $TESTCASES eq "all") { 2709*6236dae4SAndroid Build Coastguard Worker # Get all commands and find out their test numbers 2710*6236dae4SAndroid Build Coastguard Worker opendir(DIR, $TESTDIR) || die "can't opendir $TESTDIR: $!"; 2711*6236dae4SAndroid Build Coastguard Worker my @cmds = grep { /^test([0-9]+)$/ && -f "$TESTDIR/$_" } readdir(DIR); 2712*6236dae4SAndroid Build Coastguard Worker closedir(DIR); 2713*6236dae4SAndroid Build Coastguard Worker 2714*6236dae4SAndroid Build Coastguard Worker $TESTCASES=""; # start with no test cases 2715*6236dae4SAndroid Build Coastguard Worker 2716*6236dae4SAndroid Build Coastguard Worker # cut off everything but the digits 2717*6236dae4SAndroid Build Coastguard Worker for(@cmds) { 2718*6236dae4SAndroid Build Coastguard Worker $_ =~ s/[a-z\/\.]*//g; 2719*6236dae4SAndroid Build Coastguard Worker } 2720*6236dae4SAndroid Build Coastguard Worker # sort the numbers from low to high 2721*6236dae4SAndroid Build Coastguard Worker foreach my $n (sort { $a <=> $b } @cmds) { 2722*6236dae4SAndroid Build Coastguard Worker if($disabled{$n}) { 2723*6236dae4SAndroid Build Coastguard Worker # skip disabled test cases 2724*6236dae4SAndroid Build Coastguard Worker my $why = "configured as DISABLED"; 2725*6236dae4SAndroid Build Coastguard Worker $skipped{$why}++; 2726*6236dae4SAndroid Build Coastguard Worker $teststat[$n]=$why; # store reason for this test case 2727*6236dae4SAndroid Build Coastguard Worker next; 2728*6236dae4SAndroid Build Coastguard Worker } 2729*6236dae4SAndroid Build Coastguard Worker $TESTCASES .= " $n"; 2730*6236dae4SAndroid Build Coastguard Worker } 2731*6236dae4SAndroid Build Coastguard Worker} 2732*6236dae4SAndroid Build Coastguard Workerelse { 2733*6236dae4SAndroid Build Coastguard Worker my $verified=""; 2734*6236dae4SAndroid Build Coastguard Worker for(split(" ", $TESTCASES)) { 2735*6236dae4SAndroid Build Coastguard Worker if (-e "$TESTDIR/test$_") { 2736*6236dae4SAndroid Build Coastguard Worker $verified.="$_ "; 2737*6236dae4SAndroid Build Coastguard Worker } 2738*6236dae4SAndroid Build Coastguard Worker } 2739*6236dae4SAndroid Build Coastguard Worker if($verified eq "") { 2740*6236dae4SAndroid Build Coastguard Worker print "No existing test cases were specified\n"; 2741*6236dae4SAndroid Build Coastguard Worker exit; 2742*6236dae4SAndroid Build Coastguard Worker } 2743*6236dae4SAndroid Build Coastguard Worker $TESTCASES = $verified; 2744*6236dae4SAndroid Build Coastguard Worker} 2745*6236dae4SAndroid Build Coastguard Workerif($repeat) { 2746*6236dae4SAndroid Build Coastguard Worker my $s; 2747*6236dae4SAndroid Build Coastguard Worker for(1 .. $repeat) { 2748*6236dae4SAndroid Build Coastguard Worker $s .= $TESTCASES; 2749*6236dae4SAndroid Build Coastguard Worker } 2750*6236dae4SAndroid Build Coastguard Worker $TESTCASES = $s; 2751*6236dae4SAndroid Build Coastguard Worker} 2752*6236dae4SAndroid Build Coastguard Worker 2753*6236dae4SAndroid Build Coastguard Workerif($scrambleorder) { 2754*6236dae4SAndroid Build Coastguard Worker # scramble the order of the test cases 2755*6236dae4SAndroid Build Coastguard Worker my @rand; 2756*6236dae4SAndroid Build Coastguard Worker while($TESTCASES) { 2757*6236dae4SAndroid Build Coastguard Worker my @all = split(/ +/, $TESTCASES); 2758*6236dae4SAndroid Build Coastguard Worker if(!$all[0]) { 2759*6236dae4SAndroid Build Coastguard Worker # if the first is blank, shift away it 2760*6236dae4SAndroid Build Coastguard Worker shift @all; 2761*6236dae4SAndroid Build Coastguard Worker } 2762*6236dae4SAndroid Build Coastguard Worker my $r = rand @all; 2763*6236dae4SAndroid Build Coastguard Worker push @rand, $all[$r]; 2764*6236dae4SAndroid Build Coastguard Worker $all[$r]=""; 2765*6236dae4SAndroid Build Coastguard Worker $TESTCASES = join(" ", @all); 2766*6236dae4SAndroid Build Coastguard Worker } 2767*6236dae4SAndroid Build Coastguard Worker $TESTCASES = join(" ", @rand); 2768*6236dae4SAndroid Build Coastguard Worker} 2769*6236dae4SAndroid Build Coastguard Worker 2770*6236dae4SAndroid Build Coastguard Worker# Display the contents of the given file. Line endings are canonicalized 2771*6236dae4SAndroid Build Coastguard Worker# and excessively long files are elided 2772*6236dae4SAndroid Build Coastguard Workersub displaylogcontent { 2773*6236dae4SAndroid Build Coastguard Worker my ($file)=@_; 2774*6236dae4SAndroid Build Coastguard Worker if(open(my $single, "<", "$file")) { 2775*6236dae4SAndroid Build Coastguard Worker my $linecount = 0; 2776*6236dae4SAndroid Build Coastguard Worker my $truncate; 2777*6236dae4SAndroid Build Coastguard Worker my @tail; 2778*6236dae4SAndroid Build Coastguard Worker while(my $string = <$single>) { 2779*6236dae4SAndroid Build Coastguard Worker $string =~ s/\r\n/\n/g; 2780*6236dae4SAndroid Build Coastguard Worker $string =~ s/[\r\f\032]/\n/g; 2781*6236dae4SAndroid Build Coastguard Worker $string .= "\n" unless ($string =~ /\n$/); 2782*6236dae4SAndroid Build Coastguard Worker $string =~ tr/\n//; 2783*6236dae4SAndroid Build Coastguard Worker for my $line (split(m/\n/, $string)) { 2784*6236dae4SAndroid Build Coastguard Worker $line =~ s/\s*\!$//; 2785*6236dae4SAndroid Build Coastguard Worker if ($truncate) { 2786*6236dae4SAndroid Build Coastguard Worker push @tail, " $line\n"; 2787*6236dae4SAndroid Build Coastguard Worker } else { 2788*6236dae4SAndroid Build Coastguard Worker logmsg " $line\n"; 2789*6236dae4SAndroid Build Coastguard Worker } 2790*6236dae4SAndroid Build Coastguard Worker $linecount++; 2791*6236dae4SAndroid Build Coastguard Worker $truncate = $linecount > 1200; 2792*6236dae4SAndroid Build Coastguard Worker } 2793*6236dae4SAndroid Build Coastguard Worker } 2794*6236dae4SAndroid Build Coastguard Worker close($single); 2795*6236dae4SAndroid Build Coastguard Worker if(@tail) { 2796*6236dae4SAndroid Build Coastguard Worker my $tailshow = 200; 2797*6236dae4SAndroid Build Coastguard Worker my $tailskip = 0; 2798*6236dae4SAndroid Build Coastguard Worker my $tailtotal = scalar @tail; 2799*6236dae4SAndroid Build Coastguard Worker if($tailtotal > $tailshow) { 2800*6236dae4SAndroid Build Coastguard Worker $tailskip = $tailtotal - $tailshow; 2801*6236dae4SAndroid Build Coastguard Worker logmsg "=== File too long: $tailskip lines omitted here\n"; 2802*6236dae4SAndroid Build Coastguard Worker } 2803*6236dae4SAndroid Build Coastguard Worker for($tailskip .. $tailtotal-1) { 2804*6236dae4SAndroid Build Coastguard Worker logmsg "$tail[$_]"; 2805*6236dae4SAndroid Build Coastguard Worker } 2806*6236dae4SAndroid Build Coastguard Worker } 2807*6236dae4SAndroid Build Coastguard Worker } 2808*6236dae4SAndroid Build Coastguard Worker} 2809*6236dae4SAndroid Build Coastguard Worker 2810*6236dae4SAndroid Build Coastguard Workersub displaylogs { 2811*6236dae4SAndroid Build Coastguard Worker my ($runnerid, $testnum)=@_; 2812*6236dae4SAndroid Build Coastguard Worker my $logdir = getrunnerlogdir($runnerid); 2813*6236dae4SAndroid Build Coastguard Worker opendir(DIR, "$logdir") || 2814*6236dae4SAndroid Build Coastguard Worker die "can't open dir: $!"; 2815*6236dae4SAndroid Build Coastguard Worker my @logs = readdir(DIR); 2816*6236dae4SAndroid Build Coastguard Worker closedir(DIR); 2817*6236dae4SAndroid Build Coastguard Worker 2818*6236dae4SAndroid Build Coastguard Worker logmsg "== Contents of files in the $logdir/ dir after test $testnum\n"; 2819*6236dae4SAndroid Build Coastguard Worker foreach my $log (sort @logs) { 2820*6236dae4SAndroid Build Coastguard Worker if($log =~ /\.(\.|)$/) { 2821*6236dae4SAndroid Build Coastguard Worker next; # skip "." and ".." 2822*6236dae4SAndroid Build Coastguard Worker } 2823*6236dae4SAndroid Build Coastguard Worker if($log =~ /^\.nfs/) { 2824*6236dae4SAndroid Build Coastguard Worker next; # skip ".nfs" 2825*6236dae4SAndroid Build Coastguard Worker } 2826*6236dae4SAndroid Build Coastguard Worker if(($log eq "memdump") || ($log eq "core")) { 2827*6236dae4SAndroid Build Coastguard Worker next; # skip "memdump" and "core" 2828*6236dae4SAndroid Build Coastguard Worker } 2829*6236dae4SAndroid Build Coastguard Worker if((-d "$logdir/$log") || (! -s "$logdir/$log")) { 2830*6236dae4SAndroid Build Coastguard Worker next; # skip directory and empty files 2831*6236dae4SAndroid Build Coastguard Worker } 2832*6236dae4SAndroid Build Coastguard Worker if(($log =~ /^stdout\d+/) && ($log !~ /^stdout$testnum/)) { 2833*6236dae4SAndroid Build Coastguard Worker next; # skip stdoutNnn of other tests 2834*6236dae4SAndroid Build Coastguard Worker } 2835*6236dae4SAndroid Build Coastguard Worker if(($log =~ /^stderr\d+/) && ($log !~ /^stderr$testnum/)) { 2836*6236dae4SAndroid Build Coastguard Worker next; # skip stderrNnn of other tests 2837*6236dae4SAndroid Build Coastguard Worker } 2838*6236dae4SAndroid Build Coastguard Worker if(($log =~ /^upload\d+/) && ($log !~ /^upload$testnum/)) { 2839*6236dae4SAndroid Build Coastguard Worker next; # skip uploadNnn of other tests 2840*6236dae4SAndroid Build Coastguard Worker } 2841*6236dae4SAndroid Build Coastguard Worker if(($log =~ /^curl\d+\.out/) && ($log !~ /^curl$testnum\.out/)) { 2842*6236dae4SAndroid Build Coastguard Worker next; # skip curlNnn.out of other tests 2843*6236dae4SAndroid Build Coastguard Worker } 2844*6236dae4SAndroid Build Coastguard Worker if(($log =~ /^test\d+\.txt/) && ($log !~ /^test$testnum\.txt/)) { 2845*6236dae4SAndroid Build Coastguard Worker next; # skip testNnn.txt of other tests 2846*6236dae4SAndroid Build Coastguard Worker } 2847*6236dae4SAndroid Build Coastguard Worker if(($log =~ /^file\d+\.txt/) && ($log !~ /^file$testnum\.txt/)) { 2848*6236dae4SAndroid Build Coastguard Worker next; # skip fileNnn.txt of other tests 2849*6236dae4SAndroid Build Coastguard Worker } 2850*6236dae4SAndroid Build Coastguard Worker if(($log =~ /^netrc\d+/) && ($log !~ /^netrc$testnum/)) { 2851*6236dae4SAndroid Build Coastguard Worker next; # skip netrcNnn of other tests 2852*6236dae4SAndroid Build Coastguard Worker } 2853*6236dae4SAndroid Build Coastguard Worker if(($log =~ /^trace\d+/) && ($log !~ /^trace$testnum/)) { 2854*6236dae4SAndroid Build Coastguard Worker next; # skip traceNnn of other tests 2855*6236dae4SAndroid Build Coastguard Worker } 2856*6236dae4SAndroid Build Coastguard Worker if(($log =~ /^valgrind\d+/) && ($log !~ /^valgrind$testnum(?:\..*)?$/)) { 2857*6236dae4SAndroid Build Coastguard Worker next; # skip valgrindNnn of other tests 2858*6236dae4SAndroid Build Coastguard Worker } 2859*6236dae4SAndroid Build Coastguard Worker if(($log =~ /^test$testnum$/)) { 2860*6236dae4SAndroid Build Coastguard Worker next; # skip test$testnum since it can be very big 2861*6236dae4SAndroid Build Coastguard Worker } 2862*6236dae4SAndroid Build Coastguard Worker logmsg "=== Start of file $log\n"; 2863*6236dae4SAndroid Build Coastguard Worker displaylogcontent("$logdir/$log"); 2864*6236dae4SAndroid Build Coastguard Worker logmsg "=== End of file $log\n"; 2865*6236dae4SAndroid Build Coastguard Worker } 2866*6236dae4SAndroid Build Coastguard Worker} 2867*6236dae4SAndroid Build Coastguard Worker 2868*6236dae4SAndroid Build Coastguard Worker####################################################################### 2869*6236dae4SAndroid Build Coastguard Worker# Scan tests to find suitable candidates 2870*6236dae4SAndroid Build Coastguard Worker# 2871*6236dae4SAndroid Build Coastguard Worker 2872*6236dae4SAndroid Build Coastguard Workermy $failed; 2873*6236dae4SAndroid Build Coastguard Workermy $failedign; 2874*6236dae4SAndroid Build Coastguard Workermy $ok=0; 2875*6236dae4SAndroid Build Coastguard Workermy $ign=0; 2876*6236dae4SAndroid Build Coastguard Workermy $total=0; 2877*6236dae4SAndroid Build Coastguard Workermy $lasttest=0; 2878*6236dae4SAndroid Build Coastguard Workermy @at = split(" ", $TESTCASES); 2879*6236dae4SAndroid Build Coastguard Workermy $count=0; 2880*6236dae4SAndroid Build Coastguard Workermy $endwaitcnt=0; 2881*6236dae4SAndroid Build Coastguard Worker 2882*6236dae4SAndroid Build Coastguard Worker$start = time(); 2883*6236dae4SAndroid Build Coastguard Worker 2884*6236dae4SAndroid Build Coastguard Worker# scan all tests to find ones we should try to run 2885*6236dae4SAndroid Build Coastguard Workermy @runtests; 2886*6236dae4SAndroid Build Coastguard Workerforeach my $testnum (@at) { 2887*6236dae4SAndroid Build Coastguard Worker $lasttest = $testnum if($testnum > $lasttest); 2888*6236dae4SAndroid Build Coastguard Worker my ($why, $errorreturncode) = singletest_shouldrun($testnum); 2889*6236dae4SAndroid Build Coastguard Worker if($why || $listonly) { 2890*6236dae4SAndroid Build Coastguard Worker # Display test name now--test will be completely skipped later 2891*6236dae4SAndroid Build Coastguard Worker my $error = singletest_count($testnum, $why); 2892*6236dae4SAndroid Build Coastguard Worker next; 2893*6236dae4SAndroid Build Coastguard Worker } 2894*6236dae4SAndroid Build Coastguard Worker $ignoretestcodes{$testnum} = $errorreturncode; 2895*6236dae4SAndroid Build Coastguard Worker push(@runtests, $testnum); 2896*6236dae4SAndroid Build Coastguard Worker} 2897*6236dae4SAndroid Build Coastguard Workermy $totaltests = scalar(@runtests); 2898*6236dae4SAndroid Build Coastguard Worker 2899*6236dae4SAndroid Build Coastguard Workerif($listonly) { 2900*6236dae4SAndroid Build Coastguard Worker exit(0); 2901*6236dae4SAndroid Build Coastguard Worker} 2902*6236dae4SAndroid Build Coastguard Worker 2903*6236dae4SAndroid Build Coastguard Worker####################################################################### 2904*6236dae4SAndroid Build Coastguard Worker# Setup CI Test Run 2905*6236dae4SAndroid Build Coastguard Workercitest_starttestrun(); 2906*6236dae4SAndroid Build Coastguard Worker 2907*6236dae4SAndroid Build Coastguard Worker####################################################################### 2908*6236dae4SAndroid Build Coastguard Worker# Start test runners 2909*6236dae4SAndroid Build Coastguard Worker# 2910*6236dae4SAndroid Build Coastguard Workermy $numrunners = $jobs < scalar(@runtests) ? $jobs : scalar(@runtests); 2911*6236dae4SAndroid Build Coastguard Workercreaterunners($numrunners); 2912*6236dae4SAndroid Build Coastguard Worker 2913*6236dae4SAndroid Build Coastguard Worker####################################################################### 2914*6236dae4SAndroid Build Coastguard Worker# The main test-loop 2915*6236dae4SAndroid Build Coastguard Worker# 2916*6236dae4SAndroid Build Coastguard Worker# Every iteration through the loop consists of these steps: 2917*6236dae4SAndroid Build Coastguard Worker# - if the global abort flag is set, exit the loop; we are done 2918*6236dae4SAndroid Build Coastguard Worker# - if a runner is idle, start a new test on it 2919*6236dae4SAndroid Build Coastguard Worker# - if all runners are idle, exit the loop; we are done 2920*6236dae4SAndroid Build Coastguard Worker# - if a runner has a response for us, process the response 2921*6236dae4SAndroid Build Coastguard Worker 2922*6236dae4SAndroid Build Coastguard Worker# run through each candidate test and execute it 2923*6236dae4SAndroid Build Coastguard Workermy $runner_wait_cnt = 0; 2924*6236dae4SAndroid Build Coastguard Workerwhile () { 2925*6236dae4SAndroid Build Coastguard Worker # check the abort flag 2926*6236dae4SAndroid Build Coastguard Worker if($globalabort) { 2927*6236dae4SAndroid Build Coastguard Worker logmsg singletest_dumplogs(); 2928*6236dae4SAndroid Build Coastguard Worker logmsg "Aborting tests\n"; 2929*6236dae4SAndroid Build Coastguard Worker logmsg "Waiting for " . scalar((keys %runnersrunning)) . " outstanding test(s) to finish...\n"; 2930*6236dae4SAndroid Build Coastguard Worker # Wait for the last requests to complete and throw them away so 2931*6236dae4SAndroid Build Coastguard Worker # that IPC calls & responses stay in sync 2932*6236dae4SAndroid Build Coastguard Worker # TODO: send a signal to the runners to interrupt a long test 2933*6236dae4SAndroid Build Coastguard Worker foreach my $rid (keys %runnersrunning) { 2934*6236dae4SAndroid Build Coastguard Worker runnerar($rid); 2935*6236dae4SAndroid Build Coastguard Worker delete $runnersrunning{$rid}; 2936*6236dae4SAndroid Build Coastguard Worker logmsg "."; 2937*6236dae4SAndroid Build Coastguard Worker $| = 1; 2938*6236dae4SAndroid Build Coastguard Worker } 2939*6236dae4SAndroid Build Coastguard Worker logmsg "\n"; 2940*6236dae4SAndroid Build Coastguard Worker last; 2941*6236dae4SAndroid Build Coastguard Worker } 2942*6236dae4SAndroid Build Coastguard Worker 2943*6236dae4SAndroid Build Coastguard Worker # Start a new test if possible 2944*6236dae4SAndroid Build Coastguard Worker if(scalar(@runnersidle) && scalar(@runtests)) { 2945*6236dae4SAndroid Build Coastguard Worker # A runner is ready to run a test, and tests are still available to run 2946*6236dae4SAndroid Build Coastguard Worker # so start a new test. 2947*6236dae4SAndroid Build Coastguard Worker $count++; 2948*6236dae4SAndroid Build Coastguard Worker my $testnum = shift(@runtests); 2949*6236dae4SAndroid Build Coastguard Worker 2950*6236dae4SAndroid Build Coastguard Worker # pick a runner for this new test 2951*6236dae4SAndroid Build Coastguard Worker my $runnerid = pickrunner($testnum); 2952*6236dae4SAndroid Build Coastguard Worker $countforrunner{$runnerid} = $count; 2953*6236dae4SAndroid Build Coastguard Worker 2954*6236dae4SAndroid Build Coastguard Worker # Start the test 2955*6236dae4SAndroid Build Coastguard Worker my ($error, $again) = singletest($runnerid, $testnum, $countforrunner{$runnerid}, $totaltests); 2956*6236dae4SAndroid Build Coastguard Worker if($again) { 2957*6236dae4SAndroid Build Coastguard Worker # this runner is busy running a test 2958*6236dae4SAndroid Build Coastguard Worker $runnersrunning{$runnerid} = $testnum; 2959*6236dae4SAndroid Build Coastguard Worker } else { 2960*6236dae4SAndroid Build Coastguard Worker runnerready($runnerid); 2961*6236dae4SAndroid Build Coastguard Worker if($error >= 0) { 2962*6236dae4SAndroid Build Coastguard Worker # We make this simplifying assumption to avoid having to handle 2963*6236dae4SAndroid Build Coastguard Worker # $error properly here, but we must handle the case of runner 2964*6236dae4SAndroid Build Coastguard Worker # death without abending here. 2965*6236dae4SAndroid Build Coastguard Worker die "Internal error: test must not complete on first call"; 2966*6236dae4SAndroid Build Coastguard Worker } 2967*6236dae4SAndroid Build Coastguard Worker } 2968*6236dae4SAndroid Build Coastguard Worker } 2969*6236dae4SAndroid Build Coastguard Worker 2970*6236dae4SAndroid Build Coastguard Worker # See if we've completed all the tests 2971*6236dae4SAndroid Build Coastguard Worker if(!scalar(%runnersrunning)) { 2972*6236dae4SAndroid Build Coastguard Worker # No runners are running; we must be done 2973*6236dae4SAndroid Build Coastguard Worker scalar(@runtests) && die 'Internal error: still have tests to run'; 2974*6236dae4SAndroid Build Coastguard Worker last; 2975*6236dae4SAndroid Build Coastguard Worker } 2976*6236dae4SAndroid Build Coastguard Worker 2977*6236dae4SAndroid Build Coastguard Worker # See if a test runner needs attention 2978*6236dae4SAndroid Build Coastguard Worker # If we could be running more tests, don't wait so we can schedule a new 2979*6236dae4SAndroid Build Coastguard Worker # one immediately. If all runners are busy, wait a fraction of a second 2980*6236dae4SAndroid Build Coastguard Worker # for one to finish so we can still loop around to check the abort flag. 2981*6236dae4SAndroid Build Coastguard Worker my $runnerwait = scalar(@runnersidle) && scalar(@runtests) ? 0 : 1.0; 2982*6236dae4SAndroid Build Coastguard Worker my (@ridsready, $riderror) = runnerar_ready($runnerwait); 2983*6236dae4SAndroid Build Coastguard Worker if(@ridsready) { 2984*6236dae4SAndroid Build Coastguard Worker for my $ridready (@ridsready) { 2985*6236dae4SAndroid Build Coastguard Worker if($ridready && ! defined $runnersrunning{$ridready}) { 2986*6236dae4SAndroid Build Coastguard Worker # On Linux, a closed pipe still shows up as ready instead of error. 2987*6236dae4SAndroid Build Coastguard Worker # Detect this here by seeing if we are expecting it to be ready and 2988*6236dae4SAndroid Build Coastguard Worker # treat it as an error if not. 2989*6236dae4SAndroid Build Coastguard Worker logmsg "ERROR: Runner $ridready is unexpectedly ready; is probably actually dead\n"; 2990*6236dae4SAndroid Build Coastguard Worker $riderror = $ridready; 2991*6236dae4SAndroid Build Coastguard Worker undef $ridready; 2992*6236dae4SAndroid Build Coastguard Worker } 2993*6236dae4SAndroid Build Coastguard Worker if($ridready) { 2994*6236dae4SAndroid Build Coastguard Worker # This runner is ready to be serviced 2995*6236dae4SAndroid Build Coastguard Worker my $testnum = $runnersrunning{$ridready}; 2996*6236dae4SAndroid Build Coastguard Worker defined $testnum || die "Internal error: test for runner $ridready unknown"; 2997*6236dae4SAndroid Build Coastguard Worker delete $runnersrunning{$ridready}; 2998*6236dae4SAndroid Build Coastguard Worker my ($error, $again) = singletest($ridready, $testnum, $countforrunner{$ridready}, $totaltests); 2999*6236dae4SAndroid Build Coastguard Worker if($again) { 3000*6236dae4SAndroid Build Coastguard Worker # this runner is busy running a test 3001*6236dae4SAndroid Build Coastguard Worker $runnersrunning{$ridready} = $testnum; 3002*6236dae4SAndroid Build Coastguard Worker } else { 3003*6236dae4SAndroid Build Coastguard Worker # Test is complete 3004*6236dae4SAndroid Build Coastguard Worker $runner_wait_cnt = 0; 3005*6236dae4SAndroid Build Coastguard Worker runnerready($ridready); 3006*6236dae4SAndroid Build Coastguard Worker 3007*6236dae4SAndroid Build Coastguard Worker if($error < 0) { 3008*6236dae4SAndroid Build Coastguard Worker # not a test we can run 3009*6236dae4SAndroid Build Coastguard Worker next; 3010*6236dae4SAndroid Build Coastguard Worker } 3011*6236dae4SAndroid Build Coastguard Worker 3012*6236dae4SAndroid Build Coastguard Worker $total++; # number of tests we've run 3013*6236dae4SAndroid Build Coastguard Worker 3014*6236dae4SAndroid Build Coastguard Worker if($error>0) { 3015*6236dae4SAndroid Build Coastguard Worker if($error==2) { 3016*6236dae4SAndroid Build Coastguard Worker # ignored test failures 3017*6236dae4SAndroid Build Coastguard Worker $failedign .= "$testnum "; 3018*6236dae4SAndroid Build Coastguard Worker } 3019*6236dae4SAndroid Build Coastguard Worker else { 3020*6236dae4SAndroid Build Coastguard Worker $failed.= "$testnum "; 3021*6236dae4SAndroid Build Coastguard Worker } 3022*6236dae4SAndroid Build Coastguard Worker if($postmortem) { 3023*6236dae4SAndroid Build Coastguard Worker # display all files in $LOGDIR/ in a nice way 3024*6236dae4SAndroid Build Coastguard Worker displaylogs($ridready, $testnum); 3025*6236dae4SAndroid Build Coastguard Worker } 3026*6236dae4SAndroid Build Coastguard Worker if($error==2) { 3027*6236dae4SAndroid Build Coastguard Worker $ign++; # ignored test result counter 3028*6236dae4SAndroid Build Coastguard Worker } 3029*6236dae4SAndroid Build Coastguard Worker elsif(!$anyway) { 3030*6236dae4SAndroid Build Coastguard Worker # a test failed, abort 3031*6236dae4SAndroid Build Coastguard Worker logmsg "\n - abort tests\n"; 3032*6236dae4SAndroid Build Coastguard Worker undef @runtests; # empty out the remaining tests 3033*6236dae4SAndroid Build Coastguard Worker } 3034*6236dae4SAndroid Build Coastguard Worker } 3035*6236dae4SAndroid Build Coastguard Worker elsif(!$error) { 3036*6236dae4SAndroid Build Coastguard Worker $ok++; # successful test counter 3037*6236dae4SAndroid Build Coastguard Worker } 3038*6236dae4SAndroid Build Coastguard Worker } 3039*6236dae4SAndroid Build Coastguard Worker } 3040*6236dae4SAndroid Build Coastguard Worker } 3041*6236dae4SAndroid Build Coastguard Worker } 3042*6236dae4SAndroid Build Coastguard Worker if(!@ridsready && $runnerwait && !$torture && scalar(%runnersrunning)) { 3043*6236dae4SAndroid Build Coastguard Worker $runner_wait_cnt++; 3044*6236dae4SAndroid Build Coastguard Worker if($runner_wait_cnt >= 5) { 3045*6236dae4SAndroid Build Coastguard Worker my $msg = "waiting for " . scalar(%runnersrunning) . " results:"; 3046*6236dae4SAndroid Build Coastguard Worker my $sep = " "; 3047*6236dae4SAndroid Build Coastguard Worker foreach my $rid (keys %runnersrunning) { 3048*6236dae4SAndroid Build Coastguard Worker $msg .= $sep . $runnersrunning{$rid} . "[$rid]"; 3049*6236dae4SAndroid Build Coastguard Worker $sep = ", " 3050*6236dae4SAndroid Build Coastguard Worker } 3051*6236dae4SAndroid Build Coastguard Worker logmsg "$msg\n"; 3052*6236dae4SAndroid Build Coastguard Worker } 3053*6236dae4SAndroid Build Coastguard Worker if($runner_wait_cnt >= 10) { 3054*6236dae4SAndroid Build Coastguard Worker $runner_wait_cnt = 0; 3055*6236dae4SAndroid Build Coastguard Worker foreach my $rid (keys %runnersrunning) { 3056*6236dae4SAndroid Build Coastguard Worker my $testnum = $runnersrunning{$rid}; 3057*6236dae4SAndroid Build Coastguard Worker logmsg "current state of test $testnum in [$rid]:\n"; 3058*6236dae4SAndroid Build Coastguard Worker displaylogs($rid, $testnum); 3059*6236dae4SAndroid Build Coastguard Worker } 3060*6236dae4SAndroid Build Coastguard Worker } 3061*6236dae4SAndroid Build Coastguard Worker } 3062*6236dae4SAndroid Build Coastguard Worker if($riderror) { 3063*6236dae4SAndroid Build Coastguard Worker logmsg "ERROR: runner $riderror is dead! aborting test run\n"; 3064*6236dae4SAndroid Build Coastguard Worker delete $runnersrunning{$riderror} if(defined $runnersrunning{$riderror}); 3065*6236dae4SAndroid Build Coastguard Worker $globalabort = 1; 3066*6236dae4SAndroid Build Coastguard Worker } 3067*6236dae4SAndroid Build Coastguard Worker if(!scalar(@runtests) && ++$endwaitcnt == (240 + $jobs)) { 3068*6236dae4SAndroid Build Coastguard Worker # Once all tests have been scheduled on a runner at the end of a test 3069*6236dae4SAndroid Build Coastguard Worker # run, we just wait for their results to come in. If we're still 3070*6236dae4SAndroid Build Coastguard Worker # waiting after a couple of minutes ($endwaitcnt multiplied by 3071*6236dae4SAndroid Build Coastguard Worker # $runnerwait, plus $jobs because that number won't time out), display 3072*6236dae4SAndroid Build Coastguard Worker # the same test runner status as we give with a SIGUSR1. This will 3073*6236dae4SAndroid Build Coastguard Worker # likely point to a single test that has hung. 3074*6236dae4SAndroid Build Coastguard Worker logmsg "Hmmm, the tests are taking a while to finish. Here is the status:\n"; 3075*6236dae4SAndroid Build Coastguard Worker catch_usr1(); 3076*6236dae4SAndroid Build Coastguard Worker } 3077*6236dae4SAndroid Build Coastguard Worker} 3078*6236dae4SAndroid Build Coastguard Worker 3079*6236dae4SAndroid Build Coastguard Workermy $sofar = time() - $start; 3080*6236dae4SAndroid Build Coastguard Worker 3081*6236dae4SAndroid Build Coastguard Worker####################################################################### 3082*6236dae4SAndroid Build Coastguard Worker# Finish CI Test Run 3083*6236dae4SAndroid Build Coastguard Workercitest_finishtestrun(); 3084*6236dae4SAndroid Build Coastguard Worker 3085*6236dae4SAndroid Build Coastguard Worker# Tests done, stop the servers 3086*6236dae4SAndroid Build Coastguard Workerforeach my $runnerid (values %runnerids) { 3087*6236dae4SAndroid Build Coastguard Worker runnerac_stopservers($runnerid); 3088*6236dae4SAndroid Build Coastguard Worker} 3089*6236dae4SAndroid Build Coastguard Worker 3090*6236dae4SAndroid Build Coastguard Worker# Wait for servers to stop 3091*6236dae4SAndroid Build Coastguard Workermy $unexpected; 3092*6236dae4SAndroid Build Coastguard Workerforeach my $runnerid (values %runnerids) { 3093*6236dae4SAndroid Build Coastguard Worker my ($rid, $unexpect, $logs) = runnerar($runnerid); 3094*6236dae4SAndroid Build Coastguard Worker $unexpected ||= $unexpect; 3095*6236dae4SAndroid Build Coastguard Worker logmsg $logs; 3096*6236dae4SAndroid Build Coastguard Worker} 3097*6236dae4SAndroid Build Coastguard Worker 3098*6236dae4SAndroid Build Coastguard Worker# Kill the runners 3099*6236dae4SAndroid Build Coastguard Worker# There is a race condition here since we don't know exactly when the runners 3100*6236dae4SAndroid Build Coastguard Worker# have each finished shutting themselves down, but we're about to exit so it 3101*6236dae4SAndroid Build Coastguard Worker# doesn't make much difference. 3102*6236dae4SAndroid Build Coastguard Workerforeach my $runnerid (values %runnerids) { 3103*6236dae4SAndroid Build Coastguard Worker runnerac_shutdown($runnerid); 3104*6236dae4SAndroid Build Coastguard Worker sleep 0; # give runner a context switch so it can shut itself down 3105*6236dae4SAndroid Build Coastguard Worker} 3106*6236dae4SAndroid Build Coastguard Worker 3107*6236dae4SAndroid Build Coastguard Workermy $numskipped = %skipped ? sum values %skipped : 0; 3108*6236dae4SAndroid Build Coastguard Workermy $all = $total + $numskipped; 3109*6236dae4SAndroid Build Coastguard Worker 3110*6236dae4SAndroid Build Coastguard Workerruntimestats($lasttest); 3111*6236dae4SAndroid Build Coastguard Worker 3112*6236dae4SAndroid Build Coastguard Workerif($all) { 3113*6236dae4SAndroid Build Coastguard Worker logmsg "TESTDONE: $all tests were considered during ". 3114*6236dae4SAndroid Build Coastguard Worker sprintf("%.0f", $sofar) ." seconds.\n"; 3115*6236dae4SAndroid Build Coastguard Worker} 3116*6236dae4SAndroid Build Coastguard Worker 3117*6236dae4SAndroid Build Coastguard Workerif(%skipped && !$short) { 3118*6236dae4SAndroid Build Coastguard Worker my $s=0; 3119*6236dae4SAndroid Build Coastguard Worker # Temporary hash to print the restraints sorted by the number 3120*6236dae4SAndroid Build Coastguard Worker # of their occurrences 3121*6236dae4SAndroid Build Coastguard Worker my %restraints; 3122*6236dae4SAndroid Build Coastguard Worker logmsg "TESTINFO: $numskipped tests were skipped due to these restraints:\n"; 3123*6236dae4SAndroid Build Coastguard Worker 3124*6236dae4SAndroid Build Coastguard Worker for(keys %skipped) { 3125*6236dae4SAndroid Build Coastguard Worker my $r = $_; 3126*6236dae4SAndroid Build Coastguard Worker my $skip_count = $skipped{$r}; 3127*6236dae4SAndroid Build Coastguard Worker my $log_line = sprintf("TESTINFO: \"%s\" %d time%s (", $r, $skip_count, 3128*6236dae4SAndroid Build Coastguard Worker ($skip_count == 1) ? "" : "s"); 3129*6236dae4SAndroid Build Coastguard Worker 3130*6236dae4SAndroid Build Coastguard Worker # now gather all test case numbers that had this reason for being 3131*6236dae4SAndroid Build Coastguard Worker # skipped 3132*6236dae4SAndroid Build Coastguard Worker my $c=0; 3133*6236dae4SAndroid Build Coastguard Worker my $max = 9; 3134*6236dae4SAndroid Build Coastguard Worker for(0 .. scalar @teststat) { 3135*6236dae4SAndroid Build Coastguard Worker my $t = $_; 3136*6236dae4SAndroid Build Coastguard Worker if($teststat[$t] && ($teststat[$t] eq $r)) { 3137*6236dae4SAndroid Build Coastguard Worker if($c < $max) { 3138*6236dae4SAndroid Build Coastguard Worker $log_line .= ", " if($c); 3139*6236dae4SAndroid Build Coastguard Worker $log_line .= $t; 3140*6236dae4SAndroid Build Coastguard Worker } 3141*6236dae4SAndroid Build Coastguard Worker $c++; 3142*6236dae4SAndroid Build Coastguard Worker } 3143*6236dae4SAndroid Build Coastguard Worker } 3144*6236dae4SAndroid Build Coastguard Worker if($c > $max) { 3145*6236dae4SAndroid Build Coastguard Worker $log_line .= " and ".($c-$max)." more"; 3146*6236dae4SAndroid Build Coastguard Worker } 3147*6236dae4SAndroid Build Coastguard Worker $log_line .= ")\n"; 3148*6236dae4SAndroid Build Coastguard Worker $restraints{$log_line} = $skip_count; 3149*6236dae4SAndroid Build Coastguard Worker } 3150*6236dae4SAndroid Build Coastguard Worker foreach my $log_line (sort {$restraints{$b} <=> $restraints{$a} || uc($a) cmp uc($b)} keys %restraints) { 3151*6236dae4SAndroid Build Coastguard Worker logmsg $log_line; 3152*6236dae4SAndroid Build Coastguard Worker } 3153*6236dae4SAndroid Build Coastguard Worker} 3154*6236dae4SAndroid Build Coastguard Worker 3155*6236dae4SAndroid Build Coastguard Workersub testnumdetails { 3156*6236dae4SAndroid Build Coastguard Worker my ($desc, $numlist) = @_; 3157*6236dae4SAndroid Build Coastguard Worker foreach my $testnum (split(' ', $numlist)) { 3158*6236dae4SAndroid Build Coastguard Worker if(!loadtest("${TESTDIR}/test${testnum}")) { 3159*6236dae4SAndroid Build Coastguard Worker my @info_keywords = getpart("info", "keywords"); 3160*6236dae4SAndroid Build Coastguard Worker my $testname = (getpart("client", "name"))[0]; 3161*6236dae4SAndroid Build Coastguard Worker chomp $testname; 3162*6236dae4SAndroid Build Coastguard Worker logmsg "$desc $testnum: '$testname'"; 3163*6236dae4SAndroid Build Coastguard Worker my $first = 1; 3164*6236dae4SAndroid Build Coastguard Worker for my $k (@info_keywords) { 3165*6236dae4SAndroid Build Coastguard Worker chomp $k; 3166*6236dae4SAndroid Build Coastguard Worker my $sep = ($first == 1) ? " " : ", "; 3167*6236dae4SAndroid Build Coastguard Worker logmsg "$sep$k"; 3168*6236dae4SAndroid Build Coastguard Worker $first = 0; 3169*6236dae4SAndroid Build Coastguard Worker } 3170*6236dae4SAndroid Build Coastguard Worker logmsg "\n"; 3171*6236dae4SAndroid Build Coastguard Worker } 3172*6236dae4SAndroid Build Coastguard Worker } 3173*6236dae4SAndroid Build Coastguard Worker} 3174*6236dae4SAndroid Build Coastguard Worker 3175*6236dae4SAndroid Build Coastguard Workerif($total) { 3176*6236dae4SAndroid Build Coastguard Worker if($passedign) { 3177*6236dae4SAndroid Build Coastguard Worker my $sorted = numsortwords($passedign); 3178*6236dae4SAndroid Build Coastguard Worker logmsg "::group::Passed Ignored Test details\n"; 3179*6236dae4SAndroid Build Coastguard Worker testnumdetails("PASSED-IGNORED", $sorted); 3180*6236dae4SAndroid Build Coastguard Worker logmsg "IGNORED: passed tests: $sorted\n"; 3181*6236dae4SAndroid Build Coastguard Worker logmsg "::endgroup::\n"; 3182*6236dae4SAndroid Build Coastguard Worker } 3183*6236dae4SAndroid Build Coastguard Worker 3184*6236dae4SAndroid Build Coastguard Worker if($failedign) { 3185*6236dae4SAndroid Build Coastguard Worker my $sorted = numsortwords($failedign); 3186*6236dae4SAndroid Build Coastguard Worker testnumdetails("FAIL-IGNORED", $sorted); 3187*6236dae4SAndroid Build Coastguard Worker logmsg "IGNORED: failed tests: $sorted\n"; 3188*6236dae4SAndroid Build Coastguard Worker } 3189*6236dae4SAndroid Build Coastguard Worker logmsg sprintf("TESTDONE: $ok tests out of $total reported OK: %d%%\n", 3190*6236dae4SAndroid Build Coastguard Worker $ok/$total*100); 3191*6236dae4SAndroid Build Coastguard Worker 3192*6236dae4SAndroid Build Coastguard Worker if($failed && ($ok != $total)) { 3193*6236dae4SAndroid Build Coastguard Worker my $failedsorted = numsortwords($failed); 3194*6236dae4SAndroid Build Coastguard Worker logmsg "\n"; 3195*6236dae4SAndroid Build Coastguard Worker testnumdetails("FAIL", $failedsorted); 3196*6236dae4SAndroid Build Coastguard Worker logmsg "\nTESTFAIL: These test cases failed: $failedsorted\n\n"; 3197*6236dae4SAndroid Build Coastguard Worker } 3198*6236dae4SAndroid Build Coastguard Worker} 3199*6236dae4SAndroid Build Coastguard Workerelse { 3200*6236dae4SAndroid Build Coastguard Worker logmsg "\nTESTFAIL: No tests were performed\n\n"; 3201*6236dae4SAndroid Build Coastguard Worker if(scalar(keys %enabled_keywords)) { 3202*6236dae4SAndroid Build Coastguard Worker logmsg "TESTFAIL: Nothing matched these keywords: "; 3203*6236dae4SAndroid Build Coastguard Worker for(keys %enabled_keywords) { 3204*6236dae4SAndroid Build Coastguard Worker logmsg "$_ "; 3205*6236dae4SAndroid Build Coastguard Worker } 3206*6236dae4SAndroid Build Coastguard Worker logmsg "\n"; 3207*6236dae4SAndroid Build Coastguard Worker } 3208*6236dae4SAndroid Build Coastguard Worker} 3209*6236dae4SAndroid Build Coastguard Worker 3210*6236dae4SAndroid Build Coastguard Workerif(($total && (($ok+$ign) != $total)) || !$total || $unexpected) { 3211*6236dae4SAndroid Build Coastguard Worker exit 1; 3212*6236dae4SAndroid Build Coastguard Worker} 3213