From 292f6658f75d62196addc9ce2ca2d1f95a187941 Mon Sep 17 00:00:00 2001 From: Saranjeet Kaur Date: Wed, 21 Sep 2022 14:03:38 +0530 Subject: [PATCH 01/11] Create testing_in_R.Rmd --- testing_in_R.Rmd | 20 ++++++++++++++++++++ 1 file changed, 20 insertions(+) create mode 100644 testing_in_R.Rmd diff --git a/testing_in_R.Rmd b/testing_in_R.Rmd new file mode 100644 index 00000000..6b30d490 --- /dev/null +++ b/testing_in_R.Rmd @@ -0,0 +1,20 @@ +# Testing in R + +This chapter discusses about writing tests for R and extending the test suite for R. Usually, when one contributes a patch, one might also want to contribute tests that are able to cover the new code. + +## When and why to write a test? + +Whenever you add new functions to R base or any of the packages distributed with R, it is beneficial to add test(s) corresponding to the new code. While doing so it is essential to check whether `make test-Specific` still works with the new code included. In particular, check whether `cd tests; make no-segfault.Rout` work on a standalone computer without the requirement of an interactive user intervention. + +If the new code requires GUI interaction or accesses the Internet, then it is essential to add its name to the `stop list` in `tests/no-segfault.Rin`. + +## Writing tests for R (adopted from the pre release test chapter, remove this section from there) + +Writing tests for R is much like writing tests for your own code. Tests need to be thorough, fast, isolated, consistently repeatable, and as simple as possible. + +When you are adding tests to an existing test file, it is also recommended that you study the other tests in that file; it will teach you which precautions you have to take to make your tests robust and portable. We try to have tests both for normal behaviour and for error conditions. Tests live in the `tests` directory. + +## Benchmarks + +Benchmarking is useful to test that a change does not degrade performance. + From 97a57f34223745ad5a1ddc2630d55b94fe9d9357 Mon Sep 17 00:00:00 2001 From: Saranjeet Kaur Date: Thu, 20 Oct 2022 11:43:18 +0530 Subject: [PATCH 02/11] Update testing_in_R.Rmd Co-authored-by: Heather Turner --- testing_in_R.Rmd | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/testing_in_R.Rmd b/testing_in_R.Rmd index 6b30d490..0a691681 100644 --- a/testing_in_R.Rmd +++ b/testing_in_R.Rmd @@ -4,7 +4,7 @@ This chapter discusses about writing tests for R and extending the test suite fo ## When and why to write a test? -Whenever you add new functions to R base or any of the packages distributed with R, it is beneficial to add test(s) corresponding to the new code. While doing so it is essential to check whether `make test-Specific` still works with the new code included. In particular, check whether `cd tests; make no-segfault.Rout` work on a standalone computer without the requirement of an interactive user intervention. +Whenever you add new functionality to the core code or any of the packages distributed with R, it is beneficial to add test(s) corresponding to the new code. While doing so it is essential to check whether `make test-Specific` still works with the new code included. In particular, check whether `cd tests; make no-segfault.Rout` work on a standalone computer without the requirement of an interactive user intervention. If the new code requires GUI interaction or accesses the Internet, then it is essential to add its name to the `stop list` in `tests/no-segfault.Rin`. From fc68dd41d0d24a00ab5e5af15c98dc580679b51c Mon Sep 17 00:00:00 2001 From: SaranjeetKaur Date: Thu, 20 Oct 2022 18:02:17 +0530 Subject: [PATCH 03/11] expand testing R chapter --- 08-testing_in_R.Rmd | 59 ++++++++++++++++++ ...d => 09-testing_pre_release_R_versions.Rmd | 9 --- ...Developers.Rmd => 10-R_Core_Developers.Rmd | 0 ...o_get_help.Rmd => 11-where_to_get_help.Rmd | 0 ...ments.Rmd => 12-news_and_announcements.Rmd | 0 ...eloper_tools.Rmd => 13-developer_tools.Rmd | 0 img/structure-spatial.png | Bin 0 -> 8209 bytes testing_in_R.Rmd | 20 ------ 8 files changed, 59 insertions(+), 29 deletions(-) create mode 100644 08-testing_in_R.Rmd rename 08-testing_pre_release_R_versions.Rmd => 09-testing_pre_release_R_versions.Rmd (77%) rename 09-R_Core_Developers.Rmd => 10-R_Core_Developers.Rmd (100%) rename 10-where_to_get_help.Rmd => 11-where_to_get_help.Rmd (100%) rename 11-news_and_announcements.Rmd => 12-news_and_announcements.Rmd (100%) rename 12-developer_tools.Rmd => 13-developer_tools.Rmd (100%) create mode 100644 img/structure-spatial.png delete mode 100644 testing_in_R.Rmd diff --git a/08-testing_in_R.Rmd b/08-testing_in_R.Rmd new file mode 100644 index 00000000..eea980e3 --- /dev/null +++ b/08-testing_in_R.Rmd @@ -0,0 +1,59 @@ +# Testing in R {#TestR} + +This chapter discusses about writing tests for R and extending the test suite for R. Usually, when one contributes a patch, one might also want to contribute tests that are able to cover the new code. + +## When and why to write a test? + +Whenever you add new functionality to the core code or any of the packages distributed with R, it is beneficial to add test(s) corresponding to the new code. While doing so it is essential to check whether `make test-Specific` still works with the new code included. In particular, check whether `cd tests; make no-segfault.Rout` work on a standalone computer without the requirement of an interactive user intervention. + +If the new code requires GUI interaction or accesses the Internet, then it is essential to add its name to the `stop list` in `tests/no-segfault.Rin`. + +## Writing tests for R + +Writing tests for R is much like writing tests for your own code. Tests need to be thorough, fast, isolated, consistently repeatable, and as simple as possible. + +When you are adding tests to an existing test file, it is also recommended that you study the other tests in that file; it will teach you which precautions you have to take to make your tests robust and portable. We try to have tests both for normal behaviour and for error conditions. Tests live in the `tests` directory. + +## Benchmarks + +Benchmarking is useful to test that a change does not degrade performance. + +## The `tests` directory in a R package + +When one looks at the package structure of R or the sources of an R package, it consists of the directory called `tests` among other files and directories. The image below shows the structure of the R package `spatial` with the `tests` directory highlighted with yellow colour. + +![Screenshot of the structure of the R package `spatial` with the `tests` directory highlighted with yellow colour](img/structure-spatial.png) + +The `tests` directory consists of additional package-specific test code, similar to the specific tests that come with a R distribution. The `tests` directory contains test code in a `.R` file (or `.r` file from R 3.4.0) or `.Rin` file which contains code that in turn creates the corresponding `.R` file. The test code can be provided directly to the `tests` directory in `.R`, `.r`, or `.Rin` as may be appropriate. + +Whenever the `.R` file from the `tests` directory is run, the results are written to a `.Rout` file. In some cases there exists a subdirectory called `Examples` within the `tests` directory, which contains a corresponding `packagename-Ex.Rout.save` file. The output of a new run of the `.R` file and/or from a new run of the examples is compared with the existing `.Rout` file or `packagename-Ex.Rout.save` file, respectively, with differences being reported but not causing an error. The `tests` directory is copied to the check area, and the tests are run with the copy as the working directory and with `R_LIBS` set. This ensures that the copy of the package installed during testing will be found by `library(packagename)`. + +All the package-specific tests are run in a vanilla R session without setting the random-number seed. Hence, for tests which use random numbers one needs to set the seed to obtain reproducible results. Although it would be helpful to do so in all cases in order to avoid occasional failures whenever tests are run. + +## Checking R packages + +The R package checker `R CMD check` can be used to test whether the `source` R packages are working correctly. It can be run either on one or more directories, or on compressed package `tar` archives with extension `.tar.gz`, `.tgz`, `.tar.bz2`, or `.tar.xz`. The final checks of a R package should be run on a `tar` archive prepared by `R CMD build`. This results into running a series of [checks](https://cran.r-project.org/doc/manuals/r-release/R-exts.html#Checking-and-building-packages), including running the `tests` directory, if it is available: + +- The examples provided by the package’s documentation are also run. If there is a file `tests/Examples/packagename-Ex.Rout.save`, the output of running the examples is compared to that file. Released packages are able to run their own examples. Each example is run in a clean environment. Hence, earlier examples cannot be assumed to have been run. The variables T and F are also redefined to generate an error unless they are set in the example. + +- If there are other tests specified in the `tests` directory then they are also run. Typically they will be a set of `.R` (or `.r` or `.Rin`) source files and target output files `.Rout.save`. + +The comparison between the existing file and that generated newly is in the end user\’s locale, hence, the target output files should be ASCII, if at all possible. To specify tests in a non-standard location, the command line option `--test-dir=foo` may be used. For example, unusually slow tests could be placed in `inst/slowTests` and can be run using `R CMD check --test-dir=inst/slowTests`. Similarly tests that require Oracle to be installed could be placed in `inst/testWithOracle`, tests which use random values and may occasionally fail by chance could be placed in `inst/randomTests`, etc.. + +While performing these checks, the code in package vignettes is also executed, and the vignette PDFs re-made from their sources as a check of completeness of the sources, unless the `BuildVignettes` field in the package’s `DESCRIPTION` file has a false value. If there is a target output file `.Rout.save` in the vignette source directory, the output from running the code in that vignette is compared with the target output file and any differences are reported, but not recorded in the log file. + +## Testing practices + +One must be careful of what their tests (and examples) actually test. Some bad practice seen in distributed packages include: + +- Testing the time taken by a command is not reasonable. This is because, one cannot know how fast or how heavily loaded an R platform might be. At best one can test a ratio of times, however even that is difficulty and not advisable. + +- The exact format of R messages (from R itself or from other packages) change and can be translated. Hence, it is not a good idea to test their exact format. + +- For automated checking systems, it is not worthwhile for tests to use `options(warn = 1)` for reporting. + +## See also + +1. [Testing R Code](https://cran.r-project.org/doc/manuals/r-release/R-ints.html#Testing-R-code) + +2. [Writing R Extensions](https://cran.r-project.org/doc/manuals/r-release/R-exts.html) diff --git a/08-testing_pre_release_R_versions.Rmd b/09-testing_pre_release_R_versions.Rmd similarity index 77% rename from 08-testing_pre_release_R_versions.Rmd rename to 09-testing_pre_release_R_versions.Rmd index da84fdcb..989c2fc3 100644 --- a/08-testing_pre_release_R_versions.Rmd +++ b/09-testing_pre_release_R_versions.Rmd @@ -34,12 +34,3 @@ Details of performing testing on various operating systems: * [Solaris](https://developer.r-project.org/Blog/public/2021/04/28/r-can-use-your-help-testing-r-before-release/index.html#on-solaris) -## Writing tests for R - -Writing tests for R is much like writing tests for your own code. Tests need to be thorough, fast, isolated, consistently repeatable, and as simple as possible. - -When you are adding tests to an existing test file, it is also recommended that you study the other tests in that file; it will teach you which precautions you have to take to make your tests robust and portable. We try to have tests both for normal behaviour and for error conditions. Tests live in the `tests` directory. - -## Benchmarks - -Benchmarking is useful to test that a change does not degrade performance. diff --git a/09-R_Core_Developers.Rmd b/10-R_Core_Developers.Rmd similarity index 100% rename from 09-R_Core_Developers.Rmd rename to 10-R_Core_Developers.Rmd diff --git a/10-where_to_get_help.Rmd b/11-where_to_get_help.Rmd similarity index 100% rename from 10-where_to_get_help.Rmd rename to 11-where_to_get_help.Rmd diff --git a/11-news_and_announcements.Rmd b/12-news_and_announcements.Rmd similarity index 100% rename from 11-news_and_announcements.Rmd rename to 12-news_and_announcements.Rmd diff --git a/12-developer_tools.Rmd b/13-developer_tools.Rmd similarity index 100% rename from 12-developer_tools.Rmd rename to 13-developer_tools.Rmd diff --git a/img/structure-spatial.png b/img/structure-spatial.png new file mode 100644 index 0000000000000000000000000000000000000000..0e898e3e18d7dc0fa89e9685d222c95a5c9699a7 GIT binary patch literal 8209 zcmcI}XIN8P*X>4)C{Yjva|9s>Qlu&v5D=r%n;^XxDbjl}bWssOq=QO30s%soUJOm8 z7YV%wDKYdGkP^5%dfxM%`+e{C-23C+A6a?!de++e*>lb@=2&CDQhy+SmWGK20D!ZK z3Nji1KqUzNtWKQ-XSAmvOW+@syN3LIps@Qg5xh9@_@3%L0QeXQJuruW*QZ?+^xXmA zya(k+HE9ih0s!zVMVWhAPfV64R2}Sz#x3iC)>(Z|FY&NxT@9(@U+RdmeySy-ogE%V z^XisHbjeBV7vZu33bUoGM6nL{U!Jz@ z1W+?wJsmGij;h)*lJS~$n$uR)&I@|R7EMq|9@939+MXC81wpr2|MAO;}3Zn$QJAwYTxY zjhXVAK9;0LwzAHd!vygv!0g6T4namO>w9-j3sw@5PR^g9K!Ol|)ok^IiVI3pCQxMP zMX{zF`DDZ3U}5It0CKFcx6=m7RM_(}s(ach2k0-@4@ZZf40~ z?s&AwK6Z4V@Fs1@y^P~IPfVgy_w~J(S!@~Xh3)n8$9USGCXXzx`rzV17*RKhC7cq#>dqsgJnebmg}l&jtbUrWUtA1jb9161<7~i{18tW5aUUzs|kg? zjyszN%N{)`C)aqMM^tq#pA*6??Gb1Fgy{sutHZ`Ll;lh=c{nf0xHr5W_p)>_qh@0= zw>Pr(y)t#w9f-lg)%TvXJ&ot-pHmPTjO1_LnadO<7&qyR)MCmx%H5Qex}?pcPJZF@ zi%?GO=m|I|^jGQY6Ll}PIute8E0)xIgViKIuuyKFWE$GtK6LpQ5Uf!uFxJvWva*ff zA3PrVa62Jw^(d*;_viy{f6k&spd;M(&ncPYL@$YRsRZ=&YDnIZUCFp$g_?Uv8<}Z=z;fqKP3?~##GPWRB5R{Y@}o3HgB|| zG50iY-sAmA+H%Sp#|`vr^b^LDaS!$-Cf&z9N(0$zR~8EjMjVHL@8^Hlm!T=7Ua-B& zUe|N~MFD=NouM{jrqSJ}D`t=J{^a-^}8~i9}|{YJ?z*y2N!A6%2OI zzRJ`Na&%@OqtTs)#PM)Fi&ZIi52}IGh%}li5k2V;T1-D=W@rH}u|Vu=#C&`SIJ^!* z6kaWhp$s9ur9>7W|L$;T$2jfBd71#grD?EgBL}a>UosCd%Xm@b+P25{(`xlwo%D|h zS}Oed$PaK)(zX52aPVIo>RfV&f5~3t_k|K_X)ts@PK>uefL-<~K;|X>*g2M9=v|Lf zfv=Qy4T!e=gHH#UCnQ*fY>;h#= zjaI>mky%@yR zPA^-tgX?``7e1~SDRiW<+P$bTB9ep?QBm3P zONuvvBsLITZaH{erLGDGICrIF9C}V@5LP+C@X%RSyl87Su2%PH#IO8V!d)p*eyp{2 zJd~DF^A}v;Fkm+Pe@qr7dmn<~cRhq7tH8uXRR&?$!F?Ny=Zt(TDZxKDaWZkVASXD` z3-(CfZcqA{Z4?tcp*YA#B2Eu@2)*oz`P1MbBv28e zS-I|TUsBCfu%~wAj|Pj27mc#c>A{|v5ovm8YU;@SpKTxV_Dd!M^4L@Nb2=h|ka z&LtGqC!{04_D2joTO4Q(JrkG54V4)V zwJ$En4GvUG)p;4dMecAX>ZFaX$fR9+wwz0!AhtZlt7P#MT^vnBr#?6qD%ZfId)}ap zGJM94Y^|_v(=RF*lE(`vWTQ*_8R!Q^^+fE4UC8C*JsY2y`1NS@4Ay{l% zaYb3MIldAkCaR`NfU`PbjNkp!VS!HwO>N%vR!N}m@*Cg_n?r=X0ONtzGG?G8d?VIP znZ{FGL$lrT#^geWq5w(Id`He_<=aM6VIVtoc&7l>?LVgNjY>jj4zbyG4mWKd1}N0K z4;NNsE)}?@IQ!lsTwLrzk-I<8)QU(!RR_P)%5=ZwJ@)Wdy_Ec}7ZDKUGA}{4=sjH( z2?aJ}|EWCwX5W8F5p}ROCEm#!f#un1*q{{+?%Sm~B`8&r4whiRv1iK5I^Yr@yp`36 z>+P0}Qe?gkSk6ExghIez5yFA_(@;UpQ+ZI41jEK6zD;kny^NHX7Z%p#`E~~6poq1c z6Do|cy$O>Erj1jM40Ct8J9pk|MLP5DFiV;wrq|5Zs63&HRJiL>-J4)IgQK!K_9b~U zSb0^N_h+7CHB;FhLuSBWoW`PF|6LW=_M^bzBN#ETRXz-!%frL4SAka$xuQdF(y!@J= z@Vh1JFaB_*(v+ppHy=g8g;BC}`ix+Ccz*Dd#|;hhlMc`SjsIt0k1T&>RR(OpZv9uj zUM#|!mSvfm64KN>sfpDYOI!rasn547$8qJOVB3tjHvgIc=%oc=9tn@)#=urojvD%b zzh)|ie%zh6{6nriqEOPFB;=fla|bs{!`#Np^J=O;2SjR)!EY{#1B2Xuff+Mpwjb{B z3(a6}@t*JBGA&aA)mENJ_ax@Wi6}q~BtVRsu%>jD?E`yrR#Q5?4EZ;?tS6GnlCXO&Q^w8E|l~z>>LY7I^`|$28 zztr-WWSlj|D(}!!Nf3MydwOHY5&IA@>;S4I3bOPYYQtz44*)kAN=pBqs3Ru`;{k_p zXqy}v78LjIcpmU4V3E4zZ!T) z0Y1%+&{jOo*pLDHOC5nt^;BOb2k!P|;!mozR!}KWG*IEyVilELke#nE9u#c2n45?Z z$Bj|tJHR;!Fl!y4fDr7iM2|YQicD&qh&o@~9RFOWq#2RS&IO8|3G7mvZEfyJGsKlz z;3tUNd9Tm`t+hxObbiY?EmgyXistfmnnQXB2Cj`ByLGMvdY2t+u{xTK$mZ8IHMm>1 zrcDHa=4ZKh>LA&FwbW_VLHL3CYoF5HqGrnH(v5c1xBl)UUUA*geEGw+d zM%lR|4?iWT-Q%n)tE5UKu6o+p?|9lU_V8`rz?OSwKJ+jSCtny5FjX#1OwM~tP&qKs zEQFj(3&Oi)6liO&MgC|l4N=(47kqXxV5t1bjX$E6TVrQuP5d7i2PW84-PSVaC~T|< zrU@`5+&92!bL3n4WLKu&tXc6hH|6F1(!Sv8Y%Q)&S!qLMZ=`6$Ng2-WSohVUbCkx& zX97z_+JgIw`iHPtMv0y|UAc=rZUR9s`5nHL7t$PluZWv9a3Tn#!xr2;hYa%zksaj# zYi2Q`P+Mos+c}d!>)Co)o21nXXCcjH&ad<=^fzbuOFon3|9WJy3BE@Jg{<#E>J6U^ zEvmN(oY`Y(@*0quAFS%G+*y8vWr`lLP4oSPbJtYhBx9|Op=`V(N6{pUTs#4;f1FN3 z@5)l-q2{b_B753T=ADKRpVhR9#^a>s#Br*%DiA$#U@DWJQ-f0OD*g^g<*@&e$}Q_6 z7n!GihV1ISgW1*ken38I4)7@59hKT{lG+}%FKT-nup_cDXjo57Jqzq4idWSgl8QQ3 z{imTD_Hc5}uFu-+teNs`%WA#Q)WN*iTxl9P-*3@ScY`>!TXryL=v)m2@BzYVjHFhj zv)Qw|;joQ^>d92P3cB$-ZB0X_fyQ@ss*O>E6z|#EDRLp|46wrI?n1=;DOD`_8M*DZ zY;PJ{ITc_Su=Whzsus8>vcm(6tAQu{w69j2X$@#jd7m=ULLNK;xi^E1uK(FlALE0) zBBU~`swf7u@_>ZT?sr~BWV1E3Y)HtdO#Y$sXSKVClvgX#C%e{LJtsvsT(kYfN3$`Kf!0MY!Au$pqNBV6j(&g3kOAZYT z%#s<9OOERctsyFb>9ul0JWAzKes7M}v&hQgi7|0C1UKS~)}|hKFW3cgH_8_sR8s3R zIv#T~x_F>nud~_#O^}GHlTT+w`BGFCV52O=F^=Y*$i=kF3ep~I$?XabODU6!773Cy z$sgUUk@bfoWI6E^1AT)Uo9*^RAFY${8!1e(@$XeJUd(QHZ%`NCycfQ?9ogfL_IlQW zg{lm9p*u`#HKdr%KHpz#mCd^vPYzyU5k>dcyuHqA5O-)|XC^v_7+noae$TYhfnnYXK|uJ-QjIZH@Ol9L#!gUod-r@9AFbe z*Iw+=LrD$iPvpB2PvNdvWd{)(a0lOTofF%{a9rnJA9;$zywEX+0#8e+zA2vzhWHhC z4utnxxorniH<(DZi4zOa{n>T3#bSCn^&XKxtDsq~4UE7&9|a95yEojW6}6{rm|L*% zvn%<+!X^Q?2E z9LiT({Ss%L!i<)?>Mdm(R`;$iX_Cdc!@Sv>x$6fvO6*40>y@i6(UaI_dqxv5&f#?^ zgy+!+gY}0=mgB90n=PY$b9tumLILY^b6sMA6H>xcQ%9eX?Tv|bSJMt=xyLzxbAs+J z2(>!lKg>Ddf7rm5V(BhuMPiDUT}!Z0Nb*Lawu?W`9|oL~k41oj&Sd6T&?ox;>It6v z=fvNK#aL9X&7E{PM~aA0p2rKrX>4ehy4ND5zzMwObFQdZx*4_rbGwfv50~q%Y)(4H*ORqy z)4SW-1v?#eW)nxPJ;m^`+t(EQ4+-Nt9;Ir%A*bVy8r)9&1A3h-0tSu9cl$ZB8X@i7k$}%2oS}quuab)PAHum&fCIXLSXqkjX zV+_A^#|GtOY{1|0IOP7|jvqUlDXn~TW9W@he9A7TYA?cE^I|@f#8+JkRhb|E@$k z*FY!GHSU;J#}oUezrA4Xt4Eh99ZkXhl;q7-C)OgqQ@?h+my0YxFf8wtcMp#$vGJJTt=Mr_MrZDe{ zT9A+dQ6)_ax>+Hch>qt`haKqN>v8?j#am+m?EdT?GL@{xvYMQyEK5^YfV=_KAg#W}Cz#BaF3 z&T0BRhJHC*GE;%dan`9(K^NN8AmGs)0xj2*soCoZ{=UV zca7V}JWs2pbY(dDeJQd-%2H!sfwlqynBST{H^C#BV5r59<%Z{diL!{&sv!+j1B( z`-b8({Bj?DdyMu*K$O^j^dSE>CI8+5DwZz!{+tMK0@ZfeXkMDC>*>U$!;cj2A}>m+ zYK7u{ftIja_?qImu1*eLoC(r2p`HApE_o@J5K+FRS6Y{iFeyNmrcZsG8XEgVh)hZP zx%s$$<;zd|OBHTH8xl`d8OoZi?kgP4;cWg;T4+6d%t;t<{}evos$N}05}Gqo(pX6C znCs{Eobzi}G%g4ZK0`T6d9%_0)tV;ieeow#|kj#;fNRv@#7hodWY z+*d}~VF3YZ{wbi{$mDwVSev7c#Jhf>Xvubs#58V0Ky4n|)W*{ufWA41>#Md=#FP|uTYuz! zsY=4O83`P>c2VENS z#BXlqR!#<8b|!Omr&%Q@lo+=8M4Fr^#@(XIZ{zMwA$)gUdn<3q8x0rhGhj5s^nO5f zzEIp9HOy;2s3E-Fk$8HRp`sZPBQ03G)h(4yG);fLyYqVGR=9rrKG%~i$tYRpvWK8x z$-+L%k#~JG*u9nh>9s&H^}{{`sXZm?$fLq^I2RPV^82JA9en1=iDQFR9w&%#ZuN0{ z-%^>@{z8vIYgxPhz7t;IR@HCq;dkOQj)E| zL?PxR8OR40+XbQ^xpG`r^ zjtBFY^xaUVa>^})-aYH^qzSwfW<$asOFk$a}BT99|=B`l{vaWyol9=J{7+b!tDh z`|dv(@LSnLq~pHd20Ls%h=Z5IJkZO+sy9lcos`LTD3sLoS#vxH{xW ztAeFwkyCv&i;okLFWlFX#`pB9a>{DJZ!3qg~bZ25kG-tJ? z{Nkbw;%E}>sBNYRL3q&w?Z{}Qse&1z@Fe(=>YfC8oJSr$s(CyB>=91;Te8r1I*s*j zzgHBN2Xggv*JkwH6N~W68p6&a3`))DZr`HP{7Mb71Dm1!RE&;Wt#p$rhhbZd?)N)j z$OpOpDGM$}7vkY>b&;!f%LB!{+P==&(RPPCY6mT=55cJaa$i{fcAnclHA4Ut$3MS8 j{muV>efY4HIfVB;rav|#u0Rid#s?H-AIKEmHw*q>AL$pB literal 0 HcmV?d00001 diff --git a/testing_in_R.Rmd b/testing_in_R.Rmd deleted file mode 100644 index 0a691681..00000000 --- a/testing_in_R.Rmd +++ /dev/null @@ -1,20 +0,0 @@ -# Testing in R - -This chapter discusses about writing tests for R and extending the test suite for R. Usually, when one contributes a patch, one might also want to contribute tests that are able to cover the new code. - -## When and why to write a test? - -Whenever you add new functionality to the core code or any of the packages distributed with R, it is beneficial to add test(s) corresponding to the new code. While doing so it is essential to check whether `make test-Specific` still works with the new code included. In particular, check whether `cd tests; make no-segfault.Rout` work on a standalone computer without the requirement of an interactive user intervention. - -If the new code requires GUI interaction or accesses the Internet, then it is essential to add its name to the `stop list` in `tests/no-segfault.Rin`. - -## Writing tests for R (adopted from the pre release test chapter, remove this section from there) - -Writing tests for R is much like writing tests for your own code. Tests need to be thorough, fast, isolated, consistently repeatable, and as simple as possible. - -When you are adding tests to an existing test file, it is also recommended that you study the other tests in that file; it will teach you which precautions you have to take to make your tests robust and portable. We try to have tests both for normal behaviour and for error conditions. Tests live in the `tests` directory. - -## Benchmarks - -Benchmarking is useful to test that a change does not degrade performance. - From 386646b1771a2a1d03043da4e3e209850b212807 Mon Sep 17 00:00:00 2001 From: Saranjeet Kaur Date: Fri, 2 Dec 2022 17:52:11 +0530 Subject: [PATCH 04/11] minor edits Co-authored-by: Heather Turner --- 08-testing_in_R.Rmd | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/08-testing_in_R.Rmd b/08-testing_in_R.Rmd index eea980e3..b8345db2 100644 --- a/08-testing_in_R.Rmd +++ b/08-testing_in_R.Rmd @@ -12,7 +12,7 @@ If the new code requires GUI interaction or accesses the Internet, then it is es Writing tests for R is much like writing tests for your own code. Tests need to be thorough, fast, isolated, consistently repeatable, and as simple as possible. -When you are adding tests to an existing test file, it is also recommended that you study the other tests in that file; it will teach you which precautions you have to take to make your tests robust and portable. We try to have tests both for normal behaviour and for error conditions. Tests live in the `tests` directory. +Tests live in the [`tests` directory of the R sources](https://svn.r-project.org/R/trunk/tests/). ## Benchmarks From 07d3071e9c39ae5e6edc0a8bb40a6fdf213e6bb1 Mon Sep 17 00:00:00 2001 From: Saranjeet Kaur Date: Tue, 3 Dec 2024 14:14:15 +0000 Subject: [PATCH 05/11] Apply suggestions Heather's suggestions Co-authored-by: Heather Turner --- 08-testing_in_R.Rmd | 37 +++++++++++++------------------------ 1 file changed, 13 insertions(+), 24 deletions(-) diff --git a/08-testing_in_R.Rmd b/08-testing_in_R.Rmd index b8345db2..51c33ab0 100644 --- a/08-testing_in_R.Rmd +++ b/08-testing_in_R.Rmd @@ -4,53 +4,42 @@ This chapter discusses about writing tests for R and extending the test suite fo ## When and why to write a test? -Whenever you add new functionality to the core code or any of the packages distributed with R, it is beneficial to add test(s) corresponding to the new code. While doing so it is essential to check whether `make test-Specific` still works with the new code included. In particular, check whether `cd tests; make no-segfault.Rout` work on a standalone computer without the requirement of an interactive user intervention. +Whenever you add new functionality to the core code or any of the packages distributed with R, you may be asked to add test(s) corresponding to the new code. If the new code requires GUI interaction or accesses the Internet, then it is essential to add its name to the `stop list` in `tests/no-segfault.Rin`. ## Writing tests for R -Writing tests for R is much like writing tests for your own code. Tests need to be thorough, fast, isolated, consistently repeatable, and as simple as possible. +Writing tests for R is much like writing tests for your own code. Tests need to be thorough, fast, isolated, consistently repeatable, and as simple as possible. When you are adding tests to an existing test file, it is also recommended that you study the other tests in that file; it will teach you which precautions you have to take to make your tests robust and portable. -Tests live in the [`tests` directory of the R sources](https://svn.r-project.org/R/trunk/tests/). +Tests for base R live in the [`tests` directory of the R sources](https://svn.r-project.org/R/trunk/tests/). Contributors mainly contribute to the "regression tests" which are in the files `reg-tests-1a.R`, `reg-tests-1b.R`, etc. These tests are added after fixing a bug, to ensure that future changes do not cause a regression that reintroduces the bug. If you have been asked to add tests for base functionality (something that is not in one of the add-on packages in the base distribution), you should add it to the latest regression test file, with the highest numbering (currently `reg-tests-3.R`). -## Benchmarks - -Benchmarking is useful to test that a change does not degrade performance. -## The `tests` directory in a R package +## Tests of add-on packages -When one looks at the package structure of R or the sources of an R package, it consists of the directory called `tests` among other files and directories. The image below shows the structure of the R package `spatial` with the `tests` directory highlighted with yellow colour. +The check commands (`make check` etc) run `R CMD check` on the add-on packages in the base distribution, so will run any package-specific tests. If a package has tests, they will be in the `tests` directory of the sources for that package. ![Screenshot of the structure of the R package `spatial` with the `tests` directory highlighted with yellow colour](img/structure-spatial.png) -The `tests` directory consists of additional package-specific test code, similar to the specific tests that come with a R distribution. The `tests` directory contains test code in a `.R` file (or `.r` file from R 3.4.0) or `.Rin` file which contains code that in turn creates the corresponding `.R` file. The test code can be provided directly to the `tests` directory in `.R`, `.r`, or `.Rin` as may be appropriate. +The base add-on packages use the same form of tests that are used for base R as described in the [Test files]{#test-files} section. -Whenever the `.R` file from the `tests` directory is run, the results are written to a `.Rout` file. In some cases there exists a subdirectory called `Examples` within the `tests` directory, which contains a corresponding `packagename-Ex.Rout.save` file. The output of a new run of the `.R` file and/or from a new run of the examples is compared with the existing `.Rout` file or `packagename-Ex.Rout.save` file, respectively, with differences being reported but not causing an error. The `tests` directory is copied to the check area, and the tests are run with the copy as the working directory and with `R_LIBS` set. This ensures that the copy of the package installed during testing will be found by `library(packagename)`. +When the package tests are run, the `tests` directory is copied to the check area, and the tests are run with the copy as the working directory and with `R_LIBS` set so that the copy of the package installed during testing will be found by `library(packagename)`. -All the package-specific tests are run in a vanilla R session without setting the random-number seed. Hence, for tests which use random numbers one needs to set the seed to obtain reproducible results. Although it would be helpful to do so in all cases in order to avoid occasional failures whenever tests are run. +All the package-specific tests are run in a vanilla R session without setting the random-number seed. Hence, for tests which use random numbers one needs to set the seed to obtain reproducible results. ## Checking R packages -The R package checker `R CMD check` can be used to test whether the `source` R packages are working correctly. It can be run either on one or more directories, or on compressed package `tar` archives with extension `.tar.gz`, `.tgz`, `.tar.bz2`, or `.tar.xz`. The final checks of a R package should be run on a `tar` archive prepared by `R CMD build`. This results into running a series of [checks](https://cran.r-project.org/doc/manuals/r-release/R-exts.html#Checking-and-building-packages), including running the `tests` directory, if it is available: - -- The examples provided by the package’s documentation are also run. If there is a file `tests/Examples/packagename-Ex.Rout.save`, the output of running the examples is compared to that file. Released packages are able to run their own examples. Each example is run in a clean environment. Hence, earlier examples cannot be assumed to have been run. The variables T and F are also redefined to generate an error unless they are set in the example. - -- If there are other tests specified in the `tests` directory then they are also run. Typically they will be a set of `.R` (or `.r` or `.Rin`) source files and target output files `.Rout.save`. - -The comparison between the existing file and that generated newly is in the end user\’s locale, hence, the target output files should be ASCII, if at all possible. To specify tests in a non-standard location, the command line option `--test-dir=foo` may be used. For example, unusually slow tests could be placed in `inst/slowTests` and can be run using `R CMD check --test-dir=inst/slowTests`. Similarly tests that require Oracle to be installed could be placed in `inst/testWithOracle`, tests which use random values and may occasionally fail by chance could be placed in `inst/randomTests`, etc.. - -While performing these checks, the code in package vignettes is also executed, and the vignette PDFs re-made from their sources as a check of completeness of the sources, unless the `BuildVignettes` field in the package’s `DESCRIPTION` file has a false value. If there is a target output file `.Rout.save` in the vignette source directory, the output from running the code in that vignette is compared with the target output file and any differences are reported, but not recorded in the log file. +For more detail on the package checker `R CMD check` see the [Checking packages](https://cran.r-project.org/doc/manuals/r-devel/R-exts.html#Checking-packages-1) section of the Writing R Extensions manual. ## Testing practices -One must be careful of what their tests (and examples) actually test. Some bad practice seen in distributed packages include: +Tests for R should follow the general good practice documented in the [Writing R Extensions](https://cran.r-project.org/doc/manuals/r-devel/R-exts.html) manual. In particular: -- Testing the time taken by a command is not reasonable. This is because, one cannot know how fast or how heavily loaded an R platform might be. At best one can test a ratio of times, however even that is difficulty and not advisable. +- Testing the time taken by a command is not reasonable. This is because, one cannot know how fast or how heavily loaded an R platform might be. At best one can test a ratio of times, however even that is difficult and not advisable. -- The exact format of R messages (from R itself or from other packages) change and can be translated. Hence, it is not a good idea to test their exact format. +- The exact format of messages from R or from add-on packages can change and may be translated. Hence, it is not a good idea to test their exact format. -- For automated checking systems, it is not worthwhile for tests to use `options(warn = 1)` for reporting. +- For tests comparing against reference output, use `options(warn = 1)` to print warnings as they occur or `options(warn = 2)` to turn off warnings. In the latter case `tools::assertWarning()` may be used to assert when a warning is expected, see for example [tests/d-p-q-r-tst-2.R](https://svn.r-project.org/R/trunk/tests/d-p-q-r-tests.R) ## See also From 29fc6b19fe43531f5d119e8066929e74857a1028 Mon Sep 17 00:00:00 2001 From: Saranjeet Kaur Date: Tue, 3 Dec 2024 14:14:50 +0000 Subject: [PATCH 06/11] Update 08-testing_in_R.Rmd Co-authored-by: Heather Turner --- 08-testing_in_R.Rmd | 23 +++++++++++++++++++++++ 1 file changed, 23 insertions(+) diff --git a/08-testing_in_R.Rmd b/08-testing_in_R.Rmd index 51c33ab0..e0a54a4a 100644 --- a/08-testing_in_R.Rmd +++ b/08-testing_in_R.Rmd @@ -13,7 +13,30 @@ If the new code requires GUI interaction or accesses the Internet, then it is es Writing tests for R is much like writing tests for your own code. Tests need to be thorough, fast, isolated, consistently repeatable, and as simple as possible. When you are adding tests to an existing test file, it is also recommended that you study the other tests in that file; it will teach you which precautions you have to take to make your tests robust and portable. Tests for base R live in the [`tests` directory of the R sources](https://svn.r-project.org/R/trunk/tests/). Contributors mainly contribute to the "regression tests" which are in the files `reg-tests-1a.R`, `reg-tests-1b.R`, etc. These tests are added after fixing a bug, to ensure that future changes do not cause a regression that reintroduces the bug. If you have been asked to add tests for base functionality (something that is not in one of the add-on packages in the base distribution), you should add it to the latest regression test file, with the highest numbering (currently `reg-tests-3.R`). +## Test files +Most tests in base R are `.R` files with R code implementing one of two types of test: + +1. Tests that produce an error on failure. These often involve a call to `stopifnot()`. +2. Tests that produce output that is compared to reference output. + +If a test file `.R` includes tests that produce output for comparison, running the tests will create an `.Rout` file that should be compared to an `.Rout.save` file with the same name. The tests are usually run as part of the check commands `make check` (for a basic check of R) and `make check-all` or `make check-devel` (for a thorough check with/without the recommended packages). Alternatively you can run a test file directly, for example if you have set the environment variable `TOP_SRCDIR` as the path for the top source directory: + +```sh +cd $TOP_SRCDIR/tests +R CMD BATCH reg-tests-3.R +``` + +will run `reg-tests-3.R` and create `reg-tests-3.Rout` in the same directory. If you have added a test or made a change that means you need to updated the reference output you can create or overwrite the reference output with, e.g. + +```sh +mv reg-tests-3.Rout reg-tests-3.Rout.save +``` + +Note that only substantive changes are important - the check will not fail due to differences in the version of R, the locale, or output from `proc.time()`, so if these are the only differences the `.Rout.save` does not need to be updated. +``` + +Sometimes test `.R` files are generated from a `.Rin` file, for example `no-segfault.Rin` generates tests for functions in the base add-on packages, calling each function with a range of inputs (character, logical, `NULL`, etc). ## Tests of add-on packages From f6555db752189b2aca2481ba6f129a8ad8e51141 Mon Sep 17 00:00:00 2001 From: Saranjeet Kaur Date: Thu, 5 Dec 2024 15:56:08 +0000 Subject: [PATCH 07/11] fix typos --- 08-testing_in_R.Rmd | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/08-testing_in_R.Rmd b/08-testing_in_R.Rmd index e0a54a4a..1c752f20 100644 --- a/08-testing_in_R.Rmd +++ b/08-testing_in_R.Rmd @@ -27,7 +27,7 @@ cd $TOP_SRCDIR/tests R CMD BATCH reg-tests-3.R ``` -will run `reg-tests-3.R` and create `reg-tests-3.Rout` in the same directory. If you have added a test or made a change that means you need to updated the reference output you can create or overwrite the reference output with, e.g. +will run `reg-tests-3.R` and create `reg-tests-3.Rout` in the same directory. If you have added a test or made a change that means you need to update the reference output. You can create or overwrite the reference output with, e.g. ```sh mv reg-tests-3.Rout reg-tests-3.Rout.save From aac9d078949f412bae1261be832de844d2a97314 Mon Sep 17 00:00:00 2001 From: Saranjeet Kaur Date: Thu, 5 Dec 2024 16:31:46 +0000 Subject: [PATCH 08/11] fix formatting --- 09-testing_pre_release_R_versions.Rmd | 36 ------------------- 10-R_Core_Developers.Rmd | 35 ------------------ 11-where_to_get_help.Rmd | 45 ------------------------ 12-news_and_announcements.Rmd | 25 ------------- 13-developer_tools.Rmd | 35 ------------------ img/structure-methods.png | Bin 0 -> 26111 bytes img/structure-spatial.png | Bin 8209 -> 0 bytes 08-testing_in_R.Rmd => testing_in_r.qmd | 12 +++---- testing_pre_release_r_versions.qmd | 24 ++++--------- 9 files changed, 13 insertions(+), 199 deletions(-) delete mode 100644 09-testing_pre_release_R_versions.Rmd delete mode 100644 10-R_Core_Developers.Rmd delete mode 100644 11-where_to_get_help.Rmd delete mode 100644 12-news_and_announcements.Rmd delete mode 100644 13-developer_tools.Rmd create mode 100644 img/structure-methods.png delete mode 100644 img/structure-spatial.png rename 08-testing_in_R.Rmd => testing_in_r.qmd (86%) diff --git a/09-testing_pre_release_R_versions.Rmd b/09-testing_pre_release_R_versions.Rmd deleted file mode 100644 index 989c2fc3..00000000 --- a/09-testing_pre_release_R_versions.Rmd +++ /dev/null @@ -1,36 +0,0 @@ -# Testing Pre-release R Versions - -This chapter is inspired from the blog on [testing R before release](https://developer.r-project.org/Blog/public/2021/04/28/r-can-use-your-help-testing-r-before-release/index.html) and discusses how you can help with testing of pre-release versions of R. - -## Where to test? - -Whenever possible use a fresh package library for testing, even better would be to use [virtual machines](#VM) for the testing. This would ensure that you do not damage your existing R installation. - -### Virtual machine {#VM} - -A free Windows 10 virtual machine is provided by Microsoft (with a 90-day limit) for building, testing, and checking R packages and R itself. Package maintainers who work on Linux and MacOS can use it to test their packages on Windows. Read the [instructions](https://svn.r-project.org/R-dev-web/trunk/WindowsBuilds/winutf8/ucrt3/vm.html) on how to automatically set up the machine to check R packages. Tomas Kalibera describes the details of using virtual machine in the blog [Virtual Windows machine for checking R packages](https://developer.r-project.org/Blog/public/2021/03/18/virtual-windows-machine-for-checking-r-packages/index.html). - -## What can you test? - -You can test: - - * Your own programs. - - * Your own workflows. - - * Your special ways of installing or setting up R. - - * Things that interact with external libraries. - - * Interactive R packages. - -Details of performing testing on various operating systems: - - * [Windows](https://developer.r-project.org/Blog/public/2021/04/28/r-can-use-your-help-testing-r-before-release/index.html#on-windows) - - * [macOS](https://developer.r-project.org/Blog/public/2021/04/28/r-can-use-your-help-testing-r-before-release/index.html#on-macos) - - * [Linux](https://developer.r-project.org/Blog/public/2021/04/28/r-can-use-your-help-testing-r-before-release/index.html#on-linux) - - * [Solaris](https://developer.r-project.org/Blog/public/2021/04/28/r-can-use-your-help-testing-r-before-release/index.html#on-solaris) - diff --git a/10-R_Core_Developers.Rmd b/10-R_Core_Developers.Rmd deleted file mode 100644 index 502880aa..00000000 --- a/10-R_Core_Developers.Rmd +++ /dev/null @@ -1,35 +0,0 @@ -# R Core Developers - -This page lists the former and current members of the R Core team who have write access to the R source. - - * Brian Ripley (present) - * Deepayan Sarkar (present) - * Douglas Bates (present) - * Duncan Murdoch (up to September 2017) - * Duncan Temple Lang (present) - * Friedrich Leisch (present) - * Guido Masarotto (up to June 2003) - * Heiner Schwarte (up to October 1999) - * John Chambers (present) - * Kurt Hornik (present) - * Luke Tierney (present) - * Martin Maechler (present) - * Sebastian Meyer (present) - * Martin Morgan (up to June 2021) - * Martyn Plummer (present) - * Michael Lawrence (present) - * Paul Murrell (present) - * Peter Dalgaard (present) - * Robert Gentleman (present) - * Ross Ihaka (present) - * Seth Falcon (up to August 2015) - * Simon Urbanek (present) - * Stefano lacus (up to July 2014) - * Thomas Lumley (present) - * Tomas Kalibera (present) - * Uwe Ligges (present) - -View the [affiliations of R Core members](). We've left it up to the individual core developers to list areas of expertise (or things they are willing to maintain) if they wish. - - -The [Contributors page on the R Project website](https://www.r-project.org/contributors.html) also lists contributors, outside the R Core team, who provided invaluable help by donating code, bug fixes, and documentation. diff --git a/11-where_to_get_help.Rmd b/11-where_to_get_help.Rmd deleted file mode 100644 index 9882ffa5..00000000 --- a/11-where_to_get_help.Rmd +++ /dev/null @@ -1,45 +0,0 @@ -# Where to Get Help {#WhereToGetHelp} - -If you are working on R it is possible that you will come across an issue where you would need some assistance to solve it. If you require help, there are options available to seek assistance or get some feedback which are discussed in this chapter. If the question involves process or tool usage then please check the rest of this guide first as it should answer your question. Please make sure to search the documentation and resources to see if your question has already been addressed. If not, then ask for assistance in the appropriate forum. Many developers are volunteers and please be polite, patient, and thoughtful when requesting for feedback or help. - -## Slack - -You can discuss issues related to the development of R and learn about the process of contributing to R on the [R Contributors slack](https://r-contributors.slack.com/). There are a number of experienced developers on this slack who can answer questions and/or provide feedback. The following channels are available on the R-devel slack for help and feedback with specific areas: - - * #bugreports-for-review: Share bug reports for review/feedback before submitting to Bugzilla. - - * #core-dev-help: Getting help on anything related to R Core contribution. - - * #core-documentation: Discuss patches/improvements to R's documentation. - - * #core-translation: Discuss translating R messages, warnings, and errors into non-English languages. - - * #patches-for-review: Share patches for peer review before submitting to R Core. - -Note: You may not be able to access the history of these channels, so it cannot be used as a knowledge base of sorts. - -## Mailing lists - -There are quite a few [mailing lists](https://www.r-project.org/mail.html) for getting help with R: - - * [R-devel](https://stat.ethz.ch/mailman/listinfo/r-devel): - * Questions and discussion about development _of_ R vs. _with_ R. - * Getting help with technical programming issues, e.g. interfacing R with C/C++. - * Proposals of new functionality for R. - * Pre-testing of new versions of R. - * Enhancements and patches to the R source code and the R documentation. - * Posting examples and benchmarks. - - * [R-help](https://stat.ethz.ch/mailman/listinfo/r-help): - * Discussions about problems and solutions using R. - - * [R-package-devel](https://stat.ethz.ch/mailman/listinfo/r-package-devel): - * Getting help about package development in R. - * Learning about the package development process. - * Discussing problems developing a package (or problem in passing the R CMD check). - -Please avoid cross-posting to both the R-package-devel and the R-devel mailing lists. - -## File a bug - -If you strongly suspect you have come across a bug (be it in the build process, or in other areas), then [report it on Bugzilla](#ReportBug). diff --git a/12-news_and_announcements.Rmd b/12-news_and_announcements.Rmd deleted file mode 100644 index d75bced4..00000000 --- a/12-news_and_announcements.Rmd +++ /dev/null @@ -1,25 +0,0 @@ -# News and Announcements - -Here are some resources that can be useful to keep up with the developments in R: - -## Blogs - -The [R project](https://www.r-project.org/) maintains a list of public [blogs](https://developer.r-project.org/Blog/public/) mainly written by the R Core Development Team. Daily news about R-devel can be found [here](https://developer.r-project.org/blosxom.cgi/R-devel). - -## Conferences - -Updates about conferences actively supported or endorsed by The R Foundation can be found [here](https://www.r-project.org/conferences/). These conferences are organised by members from the R community. - -## Journal - -[The R Journal](https://journal.r-project.org/) is an open access and refereed journal featuring short to medium length articles that should be of interest to users or developers of R. It also has a news section where information on, changes in R (new features of the latest release), changes on CRAN (new add-on packages, manuals, binary contributions, mirrors, etc.), upcoming conferences, and conference reports is provided. - - -## Mailing lists - - * [R-announce](https://stat.ethz.ch/mailman/listinfo/r-announce): A moderated mailing list used for announcements by the R Core Development Team. Major announcements about the development of R and the availability of new code are made here. - * [R-packages](https://stat.ethz.ch/mailman/listinfo/r-packages): A moderated mailing list for announcements about contributed R packages (typically on [CRAN](https://cran.r-project.org/)) and similar R project extensions. - -## Twitter - -Follow [\@R_dev_news](https://twitter.com/R_dev_news) on Twitter for the [R development updates](https://developer.r-project.org/RSSfeeds.html) and [blog post](https://developer.r-project.org/Blog/public/) announcements. diff --git a/13-developer_tools.Rmd b/13-developer_tools.Rmd deleted file mode 100644 index e645c141..00000000 --- a/13-developer_tools.Rmd +++ /dev/null @@ -1,35 +0,0 @@ -# Developer Tools - -This chapter lists resources and tools which R developers may use. Here we will go over some commonly used tools that are relevant to R’s workflow. As there are several ways to accomplish these tasks, this chapter reflects methods suitable for new contributors. Experienced contributors may desire a different approach. - -## Subversion (svn) client - -Subversion (svn) is a version control system that tracks any changes made to files and directories. You can install either the TortoiseSVN (https://tortoisesvn.net/, command line tool, and Windows Explorer integration) or the SlikSVN (https://sliksvn.com/download/, just the command line tool) client. They have Windows installers and can be used from Windows cmd or RStudio terminal. - -Some resources for learning subversion commands: - -1. [Apache Subversion quick start guide](http://subversion.apache.org/quick-start) - -2. [TortoiseSVN commands](https://tortoisesvn.net/docs/nightly/TortoiseSVN_en/tsvn-cli-main.html) - -3. [SlikSVN basics](https://sliksvn.com/support/subversion-basics-using-check-out-update-check-in-commit/) - -4. [Subversion book](http://svnbook.red-bean.com/) - -## Globally search for a regular expression and print matching lines (grep) - -[grep](https://en.wikipedia.org/wiki/Grep) is a command line utility for searching plain text data sets for lines that match a regular expression. Refer the [grep manual](https://www.gnu.org/software/grep/manual/grep.html) for more commands. - -## Git - -[Git](https://en.wikipedia.org/wiki/Git) is also a version control system for tracking changes in any files and directories. View [git documentation](https://git-scm.com/doc) for learning git commands. - -## GitHub - -Some resources that are useful while using GitHub are: - -1. [Creating a pull request](https://docs.github.com/en/github/collaborating-with-issues-and-pull-requests/creating-a-pull-request) - -2. [Opening an issue from code](https://docs.github.com/en/github/managing-your-work-on-github/opening-an-issue-from-code) - -3. [Resolving a merge conflict on GitHub](https://docs.github.com/en/github/collaborating-with-issues-and-pull-requests/resolving-a-merge-conflict-on-github) diff --git a/img/structure-methods.png b/img/structure-methods.png new file mode 100644 index 0000000000000000000000000000000000000000..5a88fbfec9221f2bbcbc1a05e5e681a1da6f72c4 GIT binary patch literal 26111 zcmc$`Wmp?=yC)o=l$Ht=q?AI@;!@lx6p94b;zfgda4Chh6nB?G2^0%1!KvX%ad#)U z1|t=;oWogeCS0JCNsl-X72m`S-z;N%HJb?N(=&l?kOtBYJxy`A3-33 zXSZ$w@3f5|Hh~{_?way4AUN{b2JqqrL|R1}1gea=b7@8hyuR(CpzjU>-T#UEhu7)+ z-4X<{N>r4U*7i2po*~TC)}iX$hv+}e2gkl?lx~#H7n6T6L)rYxHbaUMEuq&IHtCccYAly5Xnrx!W|@hEl6!D|5uwN&rwqpZfWQ_g-1?tRGl<* z?Sqj>^_o8BLE6GpE#<+_~aX@!MuG|7!V`^RQIz-HAa;gLpb4zeEmQ2~m0ei%ChM>zb2CHH!|^{p z5-XLyRVF5wr#2}HxV+5#kYe1x#cN(xy&&gzvX%5)J|^ysieZ=7ZUD;~GRtcR4}>~U zV)2HwYVDmgZO*ytW!)ora`m&m%5y~1-LTOnTlopL`tJ3S1?#@{OLuH)<&xF-*j(2! z8vY9(MBN5n7^D1GJ^RMl3DEa#N#a`HD-Q;(4U; zA>6>>ISHS~zEH8-7~Z$?sndUqkM~~hAu-L#MY)C59K39`F7VjR=b~(OHJ*IN_b7Qc z=u**XwsA%L5lB2neJ?LxF+t&?u56_xFrYv3TkcQ!uDKwT#||L~#X#EJheD~@jlpgI zJob3tdz4z~w*J>_%>=Z^A=5$h>jaZR-k+hgZ!XvWwdN*)o+9D=z0eOaU9og6HU4k} zYs`h!2M*K9LENeX%aJ>ypr%(wA4N4PG^2116Ju7(cEcTpmegNi@z)oa3R<^d;SA7+vqM!c6 zuz~iTlpC#@4@k#x>xocFzI#U&$Dx>Y59A%{1B1m%Ahh-#Zafkp1bP4RVIrkbt-$eXuB1VbB|SANLsn(rM4khX~iH ziM{v2@RPVYU785=76|l268GTGwI^!-Uq0ynzskj#>l=ABC%jnkJN5skM5RmN%MW@+Ke0 zd`U#H46>wD6M*Kkl0RTp#b%)EAG%IC=`93B#6Y0g+k{e5h4Td0(;Gm^j&TChReYn! z!Uxu~`4MsJo5Ci15KzKj(f+fji_e$jnu3#``7HO>&n1=UmG`W$>n*QUF2SUiR;`1Q zHfxuwxm)A;AV^`$?A2CvGdpgb{I;w#+MYIESZp({~e4qs(o2a`q_;`cVoSa^ii zN$Ly)DB@m%{;!hy=!+f!#o^sqxK5SFgd3|mZJQ-BQNy>9ZA3%rilZ}{$p01umCYA3 zV7R|1$D}xz|AOBn;9Fs-alHwbQNueLuY+Y){btLet%)Yno|-Ya|(!_TKFk%dHo>>DE5w;`7U&d*&J!^YM^@wlziB@E28@fQIPCptkLep zAhxk<*fwZtY`|io*{Y-u>y>M%kYiE>LA?lU<R$2cx#}%r04t-7i`oNG&9=k|ocby+O_s@+s$I3Vl*ZLCs(Z;z|HrYp;11gW0 zlo+vn*$h%w;<`>VjVsLu#y~?!Ii?a+8evPaTo(FiIZ`E81zj@GZ}Ua?OP)UN1}c-S zTqG5?#HcNB#%J_O&~cXjis)`4{YT=S$fAU8`^YCJNtV6IBRBfIq#}8-_+cV8tPS49 zunMoaKvuni8vi{LDaC{rwZEdN4IeQoyhBbVAdr)5eOb&@LXLtEG)mB(-nnH0jB6mS zOMj9&A2BD>%Mo{vu`FL1(~G?6#9FG~q-r3A%{H77;SwM{cK`I zam;aHk*Ugk%&B|DxBv3&K(rh*Zzd1!2;sj~Q>C^}{Gq+(0%kNZs5uVRg}YgzxKQ=J zJFT6+fdbx>#Fjexqi$Pl`B!%WQo?JcKVgcD+=@}R^dhMCT#_lAf^Xrg1wI7XI4E)h z22M(Ra}S;Iinv(41OLVJa5S~p%AY8zg1hqz@+{u5*&2}U_}>$adR6&~Ojk=W8E#)5 z{btho!>lawofOV@p`6Cs0|Tef>rmQ{=@e_Ts+^5i#%9_^9u{9+?T;-t&1Q06RVRwof+?F_OL(kT z+`b8VNs#;C=lcA9PLN810hCH<;E@$f@i5t%c;oK^gXQdH&^L#7!VPI^Aez8XnY(!e zzfJE`av7WMI{kU(5xH0k&szQ*$6$GG8k9QmT9m3t=FAPr2bwnlwypB*7p=aQCaQso zZsyPZX_CpU)-Ey{xciD(C0+K?;zBIu=KG6>#4khPTqX#+B+B=2&7!PaeU_|DhYze>q-)7CY7xFB;S1y-3-V1rB?9t(OjaD5_ z-*Xp#v24IJx`Fl-4wS%jDQHW9XcCna<_93!_j)pPw&HPXhayxax>&_R5NJ;o6E{^i zM31E*74g(qPNs;?vk?UbZTv$!pv(E+sgC*28nOR_X7>M?555} z83tjg3RS279GhSlU4DmIJGBd;&Y_P%r{VH}SwF(R(d#*jQUN21-|}l|>ehKu(ELMr zmWJtu?SKC7epL z+@-OmpF7kUb$a4wJJFc6%&tA%{AMCt2j)H|WZBVnXXX4Djby;1y}U(8(&T?uDWnZscJeLQT`QEc?FATBVkGFE$ z7-Vvv!!^Rqqc)hzMauHWlxyZDuicjyc(l#Ht_Q2V90mHkvbx)*b~*I{@ZvDjpsUMi z$e2TNEqk!s?Z|e5YkE-`f?}vf^lF*m!P0upN=VGjR36*uiq^hVQGG$Uv6+dyX}Rf~ zWIWw#U;PG~N)Pme{K?Lexd!M53Tq4i!k0~`@KTW-;Gtqmlhb7K5X|d1AS+S!8b~gi z>N}yxxtD0C!`5r@Fib_w?A4|P42}=M_fdNolAh$D+E~bthV=!JO zgdmwW;Etb@bxuqOH^~i$8Fu-Tt<}KVA7qR~_i1>opbI|;&nym13rfBRwuoxBIXQlz zmXXVmGBvTCE&alZ%D|RyO$D_l;?Z)1ejrL>)0a_7;o5K@EPoe#SwLkwwlhAkM!Qo*-Jbd)S1=MqEd}Epbl*>RW&H`Rlj$f{1MC$Iv z7b$e!y zg3IO+Wg(;6p>;c9dJQa=M`N6T)^+h$oMm@{h#}~KWatMHFlltpb5P!++<6O+$zs0+ z{Yie?zP8|vSBER$l&rGYnMNoWZh|oHskGoby!Gnz0Wpoo&Xx$mb?H+zS#t;xNtyt$ z=m@|2*Gd9T`>_FB^IfmBK+eWyiTI61Gl`#B-aq69g??1uqs@}2fuKwv=)gu{mBLs% z`&VOBmOZHh2xeN+a0+(2oeU9s&+jRfe@FgCkaG*5Nr@@gY;G3f(Td~l#fjIi+pcw0 zvqZPI5McHL+o9`ZU?Oy=vzOE|j(^c2ro}3Eusm21Xsktp>Am6pco=mK&^fwZM zZjqGxyJGt!=;z9Sakia^OxCRsxSxYfcGq#k^qK*G%FW|VB`jV}x{{c1f4X4uti)_A zDXXTi-wrI8$N!gt)|tHsX}XEaJa-cC`8GgjE&Qv^MY8_SLj7Mtef|40`4`9b&o&o6q$jItLXIHo(fH*mr-vj( z-UQK3dUKMVWKo>iC3SURGpWs3GIjH{G~z7xvjEh|vR13UE)HHkWr8qd#y8?Y%kC{vCqr%@;XBJKWdjpI=C~zbG7f(h^ zN!fn$eLKyhdKjB=IN}Fg0Wj8?W;N1DOHB$c@ zz~q2ObGE|(%@Bcu-iaT~$b;TD0S_(0eV|`CWzcG0(tU0(91+7 zAuHOQC}7eQz(7uQA%jjuv(CqSE7>!qDaj}a_R6cPenF7z4bG1?o*U+`C{*Ja6&Zor zU+_$WGux5eoHkPmo3rtB5T3VY6k|Wo%o+@$9lZ6?CYi;R{&06wz~1V2xXz=2PidZ< zWNfi>6OD67<0i|2MOZw*&J}5ud;%Km7=j2yAFUo{{X3jA8B(mr2RU#)TIbwt#l}4o z_U4&hhy{AS*1~7Y&giVw+QYS8cKs?#*Ax6#%MXYLIOM`f2Y1UP1)sc$p=2jx(@W8H zoa_av@lJ*usdnkx>qQFoWz(UeAxK}_#vm-gd*6l$9nE ze0jta|45AtM`SCtwYD&seYjyQ1-jA6PVvjoRon92@{1|i2pqQZ?!yt8sd@&)2+gox z`?>aY4{!a0zK`Y4?m9fnyvJRF&DR2fybCY79H(sKSjFUYsiB_U>VFvN$@l;s^FgZK zYdp}rVQRpQ{{-pc|6ITS1C6AOO2~QU-ni})=_{P<-!eNlL2MY`Ku;Ypo#@Znl57_CUd#&~fsG=72b~4TrHywlM+dR#(Jen_ znU@AAYOc8lK8_~O5WYhb5Aj;Svx&_&LUx;cOfIn(#+N(Oo}Tm4Gtrp*X;tmn;~>XV zwSxCy2TFz>o%(GGgdd((3D#w{{z!WSk|hG>mR0I7CMl=z%L&y2Mz4IU*)EJ;r>mUV zWT@A2WdriQtb_rvMsza1cn6^LjFAz4qoF0uQ@YmnT3(S6VzLFmUEclN2P?M-y*~gX zxxdwx>kz9{&Mjup9$C!gW!*@pMR$~qWZDR&@Mn_eaX=;0coTKqJ%KhvW{Pk=4!kgK zy|nvo)Z#lbac5p*DM@|ru+nvLDJ2i{9&X(9VlEKVa`$@69pQf#UOiTSd4S$MKsN36 zrd30-BtSHBF&+RBu%dtr#ebi=8n-Kn|8JsOxw2wbwN!HC?3P${;CXT@t4MD;;;>%5P0JQLd`e6 zByvszMpUf>=kvg$$}$gb<3`JqwyP7d!CExB${CASE%n{F$rJqfez`(DjfB;oKK&OD zhT|CIS9U4+Rxn?J8f2ELCMYxj!Ydh2kH9})sPunDT#OHomQ7jqp)9Nx@_r*}KqL$X z#9RX-nhj8NvQdM#G_cfMrU3%lFRotdn*P-CqO)GFTSpOESk@N#``B zW=&Z5fy_l602rMR(8FYaC06|uAR!~@G|f0ttP_Ws+h*=gD0IoDdwO6!xd2rAKyzJd=VMA< z1@xLB;SK!vpK*HND)-xW=V!Hp&NIcZ+aX`8toqsRZH@@6=&(XW=f%J3WWE5-r+6KOT6592B*13CLv>?<2M+ZcxKa%2hPV-0xzp$YJ#wn z$M<N<>)~N1Ik5W?K#WnpbHC2R|aXyFUWxY4Ms(F4eE@nr*pQ z4k5V!*Fx-++Pr2HX5hMa=BjRrUjp3a8`QEC)}i8Z?@ED>K#|k9&dB7<6mn)-2(NOS z@~r0Otw9$9&6{^2#q|I=<*bMsCs@ndt|T=!haG7{ZU#9Ow&UYyWJw!93$WUV5BdpZ zkCkdWENwEsxBzcxGFhGeEw?-=r5r>IVHs_KS;xEkk^4YX?5J?49-v6}Z+-vLrWRW% z6_Y$!YoC54bt$a$=`IygM>qKM6JQ-Q3l!)!1`7mMv>?by)?BlhnsO;dU+WxTf~%%K zi$L8>)BW`+ipuKu!3vUrCi0Wo0KHs3_Ve>DbU9Suf}GoggNTO51LREt?8{?Sy|vj# z+2?G~N<;L0$%6X#cXX^;jWrYZ#VuIb){lnmbV|~97#%+vd656Y^Tu%sUz4hWO5ewv zpMH7V;aR@!@J)$~O<{Am$m$>`sQY#DV!MjQeqCU^TkM4T#S!@=``Ltv)^(;_p-b{|g z`&F~|KBE^rv!Ha0J6*u9?$Ya8%3ZfC1@6+Dz+EmrR}5ptU{4?WG&xI6G?;z!*#Z#1 zancT@ex)1iXlthP?lYonpM2G4QgkXA^|5gbZ9+qa(3|n9`ZGcEU++HOkJ67Ty|cuY z$h|AR1ouC~AC)XMZqmIzwFvhqx9E)NU1S3H2Ogs`eFp1h?yN#pb0i(36=PJmgZ5Wq z&8!8rS9+2d{Z1SJ{$1Rm;Uo_1VSK^D(-JC&G70DlUupkf!lYyP5KxOb=de7R*&{vD z8+~~~7f11tYT^r94=JyYg?KI3c>u0OH0fvtxIYWSDTK;W+H2f`)><(Aa!2^?robME zXZ(&{THK^7+E<2d%rq6|t0a7~e=bJ=54fSW7gVx4wJT~BGl9*?DHU4~i%L{Va{`Aq zrO<|xE5?hz0H!?tr0WTH$jXn85!Rp7lh~5_K^2BL7EU0-qb&Gj#NK{>xJYYNvPB%) z{}iQAFJzM7>1Y5BU+(S?toJ`VRD?6ThqncBxDSX1kYqu%8?{+&$LgYvwkA}nAiaU6 zwBNn=mlV4n{Z=ViO)MdE*_jq`zX$6{V(&>1aZ9!5w@n3x)*gRowkO%faV)eUQhc_vO;sYO%Sx7I*UPsW@WwvZyIqwAl z?vV;Rp-fJYca6_VB13jt7yCzN#l_gdalbxyACUw?kPtAgOmj7<{uRoMWa{LOm~LG< z(|w&gT+d|4+ZY9E?=Jy>^B!m9TfF}kihaGi+AEV6;H%f_11}{1F__M@24vs{Q=IUP z`%UL7zW+=E(0?#R|3hj=jR);j7&ZF;o(NBUTY1UoxTzg4uCECp0ns-5vyloqPtoO& zV5Pa_c^}}jYx3lH1R%fkZ{?rGB20yWZ^&t zFhsnv@dwN?DPMx*Y-zu@T#$6jhkWW@VJDeA0Dwj?s)kgwbwCe^_3B8Z8Y|go9N}5n zdt2j>bQkLsSJYIPPGhDlmA^J)C0S_eem_dflg(S7Dt?g`#`tF(A8vj)I zJ5VziYnmluU#DGU0`@)LV27Zx+;@V@_tIW|^vv?@soFofM#icLOvmt7b!mHixP1 z!D>GPKm@q%+|z$sPBl(d4o!k*iHRCw8iBUR@wP!>wI8cDT58M;IGhA7s;zq2^P;V}_%&@+j zB19_b<}q3nbK@4e`_PmVsAebn%OZhHxiyy zVs##ww(65n?;vn-?EGCJS4MO87^79D#H5(U3!8YJ-X0`%e#tFGWh1$?=OQJfmOah8 z)r;zbkkd#@UCaehQ|(|vA*YsLIv1<`uQhJlv#^k5lzaW}3dkJZnd5;6r1#?fb4hcy zWmM~_(x#9T7it*JEfquUd`%Rtij!-Tz%6h3^R-gOa8vF0D|Lg>091W|bOBakXo5;S zpiC|zrN_N&QXyareUI{_wy2r^LE?3jcQ@}LpRQj)v_IbgQMFX_(b=7AV;1L=0Qsjv ze#pdcX#Y1sJ8cE3DO}~y1ai))u1P35%ddG2(0_PWP(`a=BdtH#S22tCZ@_|HYZYQ0 zk*y_?T803#;1O!@>-_S$d{i8uzq+5u#(3{9Dy9uGsoWB9o_q_~7gnbUOcxS3Kc-Ac zFq8Er^8GW?qiZp`>~1^nsLN}4 z%KdH3YAwKqv+1zucw~)8aRR1=myk0(xKR=SE)ao#&Uhg^mlp(+Ky8HPhQKDuf)ySsW+ZEPLp#Bp>~X;odhEBAZSAK=ZS`!?f#rPN3k_`PpcR zmLZ!?={v~qOoQuC=$OZttF!wRC+pRP&u&(0Je$s5*{6Af^yfiGt>ROqimA-jagMC* znBQS9Kk^r8iNHVvl2`-DXTrWFkFUJQ4 zi>>asi%%x$EEbs$ewUUwvp#)r2Ax);^ORqoSdTShu-scSO&jyw_~`cy7;f;z+R86y41oi#z^9bHRoRZ6sc_p(%l_ z1d`Ma1kzG-i0Xpgi``aE9X37w&Jv+3e|}45wabwk!8bo&U-_TzGn-{!0eY9#)6sHB z4`7i|_1vAX=<()1Q)RhfS1wz#AOG^_QaoLND1`meKR?bRhB>>7Hs|iCg75LICx@qy zlV@qCotAp`i^&twY=%aVLf=otNl^E(G&6;g*5k;RL9NXNgSVBWq_zj2oE!@kJi1(7 zB7s3-gcw!Qhmj0BYk@7+DpX>A^luDr?);qwbjM6_pPs~T=`@haJh^;m$y(n@6&d@= z-`t-CTde12$jsxINJ_C;2MpkC@d#Sd8H&I2*xrMco` zJ@!hR_s@Cp;yL~3acJsqFBv*@dx1O);!2~SVdu+T!tQLQglcAg`g6(bM^kgAlQmAP zs3zp}$Q++_cicC56_MgC*IH8T%hTKR{wJFSW%neiwX?98aDJ!Z_Gf+`gLC=|GK4B; z^SQ6#qjUON(f7|yloZv%fx4c zu)1s}bdu~zJOR=Dbf@npjyXco2W!I{ybz@Lnu)lbxhzD@rx9y`liP)y1L0&nz3y<# z!1jQ}pEriR3WnVJZp(3_+oID#Ogi<@w%PI+0LEm-mCHvaP98r^y8=`P0fNo*`n6sx ziBzfiuT&XjSV-$iPom%oAYW&69DL8y;u3I4<}@`!zF~NJ_grfCha6ZLu(Bw{3#xQk z+|cMZj$8MWJQ=qrcxKFH+{#g?mLmxHyCcAp{k099RHBDXU!&sipuJg;ab1 z>c9+XJp>wdeeU>WJhTFQM~Tpzs&BDc@}c*?Zk`hyZ+u0;P=;6Z9@gKn7EUvD{m;IgwD=XXtNyhA z!uPu5Rq$o*{kKvqWezwK7_(|7e@LaR=4Wx;M?-m?$BekET}}#n$Piszj=>w5@52MoVdhD?c&k zY02<98*=Uzu_!7MK$4*IKsKDz-}EZcuV0Xe7@YQ)T%2+~798R}?gO*m%Q7%%(KUX2 zKUe3bkn=1*)(%W(mgiU4B+0TZD&lqByzyc%L-qEE(+53Mw9|R~(yuR^Eq)$+oQdBH zI^2)OU!GYnW2^_VPdcTH%Zh89Mh3sl$Oqe6J<%7vCz$t$S(Ss@#0K(*;mAGba!tG3 zW5A|fpFvNUP^bZpwwpFC5%b)N9E6X^Q|Io54Ph5H$4VF;Ge;&!wlrpZb@?1}va;Dn zriNk1uG-yBWHVF@C*^i@YQ#TZ!9rgi?%<^CrOwEB(OUaAiYZB4?|QS^!tVWjhByFk zxFxLsQa~Ci*B?=JV7*I2#|4TgazhxrN7dR`_bmNU{zj6$|aQ!LKQ7KGZN^bjQ{pC)#f1$^Z{YZU#L z723~3?`{kMv>skcz^&+P5keCYs?&n2iiIq$;yd2}WsyaTS&bjEvKdkSDZTZw-T2lH z8M1Nmd~4!+%ODK-exlB{Y-_4k$J23+QR{^Y4`1eys7jV;z)x*W4LNT2N{9VRKN*k1 zPb1b=74W98K9IAvLZWaSqe}d*L=qZt-0hfvJXHsnh`u=)+4VVqO)Tp_jpBv_YiF4x znRgR7QIE<*zo#bI(}=Cw_u|rex+H*Wj5=)eP6dKM38tDfm?@2hdQMkc=7oGpJ9!>7 zhUQ&cr{nZmYtX0)iso<` z>DU7Zsu3q}!tJ5m69bBJl0e_^`&<;Pn&XLjrEi%o=rt&J0mllQ$*1QS^EkV=k_eVt zTEU$cKYUhUyrpo#k&=x3}lJT?E9X2|4fRr{EjDg&>42}RkS!|}0Zw}dd~g%_ zr+_QR8ofc$^|$q&obFI{;|l)s!kqpUxJ5BDuVuxOr~xIqLDk#(gY6x=$(pw9B%KU? zI~xN(5*dh@ZHY$Em{c?kL#LYe8r#pXK*0MlfL*9XVBI>oRyV&-ivDG{u)_vwUs&|y zSi2?%il{2o;Ga4UT2_Y@DJ*=jt`MteCy1M@-P@#+=$o*$fz+G_`a*|px2brX$VT2z zt+RjZGFl8lVf)gkESlo@Z{$F;9;`P+UvtfRWwG4#`si$94E4Jj(VBv#Fk%YBVAwp*(}setIMHqjhFPo-YRJoVN{j;374TKX1!*%Uyh6KeTpJ$0n> zKR%-cY>w@<_CFq-LPMH11da)EtZ2l&0VU3vBPzX)r#j;3$&uTnsMFWE)PlLolj+0l zzBU8AdY@LS3P^^pzbI6TK)JBOj)&iVzxnJ3_0`1D@sVA@xtxU5+VNtX?N<22Be~*2F9>su336b9=xWh-`Xs$RvQcLhvTGk0&FF}2w)OVh)*P<4L%|B$_-;e&-#1g>>sRNTw!Vrhf5{)R zZZAGPRSZg#a4$x0)ii0dm?E{`IRyv(kz!WiN$L&x%YE$?gooTsc+rGRq{whnhCwd4 zYwQ`^tyTHR0%t^Pl~;DjPrV9q%Hf(MI|mO)l(>Tj{*2(?zQjCARNZ$Rt%>uA3{2Am zRBZN`eV4tgU7KrD62$6}T-$KnM{PDeV3r*mOtQD?68e_O8CS{BcB{II;Jy59PC&O@ z;mdt+N}M=WE{`H32{BQO(*;!CumOGmcC3|v%7ZpsjRQ9W?MluA;{%0ZHJ-$UPgU)$4~ih zRD;knZIzLQv>uwU0kO z6MXJU-fqe)BN{t{jcOLizJvf_H}V|7=Xx|2$#U<8z|mBW zf#jiaCcN3ipG%i#cifi8T_Ipb)l6Zxr9bx|;CzbXbeO3H_OPRm7q`i5cRFw$2lO8oWUO3jG7zC-@Mp z(zk)CtZoNu{Y=i#T}y+zP7Hxh*dvS=@IbfpBU9s_3aeiNew8+W3`@)|Z)F6Cdc;GTp`h#Z5n(yr|1MlYJK1*AkU(Jk_><;9UB~=G7(6V`Gto|6y;<+ zIn3kl##?sZPn?7TS-@1{9^*<*ld3)h^J3tYV_pEKv)<&u#z7MML5Dm+zQvl}_8JuM z&my);P4(LXL;BLhOh=u^Sq>>U_%HlBvH=wS0qupeJEjR?=bl_#NcXLXJ$Gzqv?^W&vKkg7lCr&h#iqmId~)`IBaIInat>uOtSbe` zLc%Z4rCQxd_FTdqa(=Yzju(c3cKt6A{yZ*S67^EpY=BG=Ulo_J#LQl)b{~>+8PH#x z$T!1q&g3+Jj;tngH*WUUAm=i2=-HkW=LU4YB*3pYt<_-%6T{IFJ^8m^(E7^(0f?1| zj+`DK{j#!AtdrT&{_`jEYiSfI!{V2t7%>sOWXSg8W#858Q<_l(zC`&<>2h6z)n)c^?3_#v_CfQ zd*#05E0Did!kr=K+^9;Y27iW`;mucD6QA_UW;4<5N#f+h0ahk(ikRQ)!X*-|LR9Qy zW>qC135hUxB9!F^qP4@Z4x2B-Z7^|$3-~_$b~eTXm!2_Pn3<;|WeeiGuPo=7tL{G& zX4zl-ouEXj^x7P@-Plut9QxjSDkL;x)-izjiy6cWcW(BT2yUD5@_nxn<6yz?57PezXXwV+ z4K~9hP5>A@zBD=^A%GU-2OPzEmU2f=e&&@Ib{3Tj09HHXY?FuSd=AcjZe98vU@T!S z`V{f#WyCE$N%8y3>R(sc!w;J zWEKti!Nw!8N{Ho3{L>EvxkfPB=ZENPlYsM{Ug{><&8u(W+|QPhdi9onUgs!8=sH?W z&hafzBMzO_`hxmE?zP1_ zjDS{;l%N~xkz3o!o3^X3MwuqO<{ITNtuabYx;P_huAszy>6_ z_z4c|j!5Rx1AM6te!o*pU!F~~jw73}h-3pwR9xH~mgUeGkKd(T-nwV|q#xafx}L7? z@3VNJa;^0OD>AagX&^p%i;(*-*j2mC2ne+)I*xKU%(eam_>PDaF1{ItB(}s9To%gj zbqbG}3?<*DAEk(SY8rXb5aVM&;>Qu9Lt}@OqryQcUJ3I@%dfQA%N>+~Rn7rO*GXM#gb&)kkaK zW|D-VATDaa%5B>Eqb60nTQ9oOr}{-aUuv2qV1$V^5Fq%|hBJY8s%7$6wLb>dcmTUk z1+a*0`LP9sPU8&&kz(r(usk_pF~8?VK&C+%ElqvrNP@Hj5wI4G^Vv||(A_;ovqSC1 z?g~BtJbClQuL#ap+4V}_HFkgahJtbnPa*8v3$aF;1h_U2v6io(6t(4oY-mQ%$Q|-S}E}@)@mPi7cQznn)Sk z`JDoD`dRS8VYojdSIv2Z_-LpjC&R6+H6Y*fR`6HV91(V$LD0&vtfBlZRVHJM#0vl= z3PL&zH2lKLpIM~y+cL`akRnH~9xxsKy*@&gx0=z+EqwGG1We{%sFj#} zFGe|sM)Hq*L|b_C4WmTw_4U!H<=mFK+xgu?M%w!Y8Au~aeXptt23EvHsnagPlkjGq z8rb%yON?p1skPHh*_xKy0H3Zhy^YZKBm_^!&O`nH~CMdeFB zCg0xmBymES?g6=qdQ>9bFQd>8DI3b!Pf_BIjbE&<`8qwQHux1Kz+)4NqF~kgF zY1dNx*|>3}=Is3>`r#hGOh<2xp#o~MHbbE)qGrlwqs z(UpYDwlQuOh*CxC%Wv*`*uwN;##AC+cfOFk_NaI!Msqpu(UZv9zcd0EShutXE1ZJc z1O*x;?snetew;f`;Nb*xpOKxs4L<8MLX~2Jq#o;JO>f&a%x#FOABP^AKHn_NajLHO zk4i>XS%vMp4H*H)k#(ij`nj|AQiuC&u8XpDO}5?fu+>N7t*d6jZ`w0}RQgwfZ~=R5 zA!Et8$V(IJD?a(`B9s17!mwM$mx+JUU>;lJFA**a*Tiauv<$6R$O+=Mc&6-l)Xn=9 z9e;tUhABI`*UaL)_*`qb9#FBQXTEw^4+D-|dQw%5g?}9*PAaI??`9BGSJRuK1VPwW z#{F(o(#GmU8jELr_f;qB(W3o7@_4H)M|LF-VI_fn04LPkYVrY>t(f&1sPl)9IgJ(6 zvbJHQgEmGkI1fkCCaqv|X1UT;4rjzZWrk|FkY0ITN6=r3Vcq!I!nvQN7E(r^@L%2r zyd^cb%aV99m%>V2sr$jONRuhwXXsBs>5yjaVvph1?}&r%x-Oy#MaF1dAY60W1e@mf z@W}db!At4e9T7uxx0(5gM~_{TU|IeY+E~SVeZu?dPd`k(GCscW)k8kH6C4_?+z|MD z6ge|QbM%CQ)5IL^SsdK-Zh^l12Gs*&V3qSqlA$jRKOR^Cu~1B4hLN{z!_m$}zmZlU zE;=}<*-r_IvjEy_i%98h1M8-`{Rq9i+*E!ul@~}ZqC#b`9X~CHMq}IbXp2|+vUO&+ z57PmlrY<=Ypj~e*P`FG9wbO~VMD7fL4eX%B+ z2>kEf58+k>An7x09EyvWkpF**oi2a({_oxf<5st*{~m7xE_r#-w6vJmvNL^#U7pfd zvg+JvaXUL8gE;?!fin!0$hhh6p|w+lT^EJ@@IYq!Y3irDKL|Ijb_!oS$x|Bbqg(-e z42=KN$53!#GrjvSA4AVSeGGBVP*D4~Ul0T0EWe`!J_stq?QBp@@wXdHSwtLg2fWZK zxLR`flCPSishTaijk72?!O36C7~BOtcnVxgqec&mgCsBy#|z=t-;F7B&!(KuGBGyp zi2X6)W^gy;^PoBCJdgdB{OlU5y1RAWY-jEb#;CZ5SIP*+swZ8nO6MWis;69 zeD9ER#Y^~|q_+jnG+~4>AdOz;*K~Q0^DZquUR&7k9HD)jBg`gxYE`?dYs@nwAaId8 z(cf_)XvE>FnBmKm>+8Zx5uHDg)ZCQ3`6(!(AbNXUj%3G-QKc~?b-x8Mc^9`;Z(_U* z;T;87_)33kDu+q4w`PHd;-L=^XZ|V7ZPGCI@A(@ zKUjTc0qx3i4&cmmPXVXlvZnN&1rG6rG69}ppg#IntKq)y=Fj!f zH{~?`jvKPdg;=%WA{{Mq?o@^x|I1uPHP8B$JF-kUcRz;poc>w@5{bW%$-6yS?1dst zGqry8%2|R}+50gS{|mSkxP^5)HA<#Lz{!((GDWl!J(=-u8Aq;5hVR-Lc-b75mVBWe zn^?-g#Wv%alu{4c;Cu@V@-gXd-dod^L&uqNxKxbKLH9G{Y-$Xjd3&|FEGCXkx8c42 zEY@*j>Kiz${?D4}-;L9s!yA0dz<1yPDNl1a`5%=v z8c>))>to{Jr$i4OSsMWPnsJ;AwPScr%bvq;u-Q z8}+)sB%*2*_6G}e^3(a9ZDU%XP4`b4pyaCkjraNkvu4QjA4>d*%>*Ko;%cT#Ku}lA zZ@1oc$eLB7-~+-|L>MP#0`ZuPv2zI02vXj)%o;R<5oy?{+Znq!c2zQ6LqBBL-feUE zU2H9i2Bv87>-w0_@{v!pyx*?(eVeLx1}7cjvvMAd@oM+C@%N zc%)@_W;cS_^89Vta<=nD-lgO|3GUl2fNx4V095TPjASCb94h2~#h^qWor6VfiD)*c zOP|fR1imQ6AF?pn+m*4M-V!-{8=wZ0YTFqL-&lz@ra!*cU@DH$XhHztPV@g!+?j_% z*+2RpJ?J6YNTdZNYs#9jmVGZ{3xguEEBn^8DkUOgNf>*UG0Bi^W>P7}G8xMlOB92# z$JhqreCBzc-}U>R>nzuG{yFDwf6R^hy65xWUN<2NhKI&xkNFMPwP7~s#ndrT?K?KC zqj}4-Bgswm!5YyLp8pz+F77V}GRzmb zD?tR~W?{EVyP9WUj#0KUl#=oUb0DuOF|otquhRU*;*P@Sq6-0&2~bc}?gJ0s+X zN$<_?eeZkl8ar7eygcg{H_PeREMytrZw_{4s_6{SO79Dm_o1Ne+eybyM8Xi6VJe*V z_h*Ftc`s7zIr^k-ejhGOzPK8yUa%FewU83b1alX>&XK$22itFwySq{E%j# z%pekUN+nsqCMvbv{?S0+Az1$GQD}+fO|p4~6P}4%5S~9oA~u2OR8lv@{O4G+UKf^K z`Lwbe=GVCcZL30zM(+v8+L$9^kV6L0Q=}IMs-VDRxK6ltEN|=#!#{{`maHUbZQKJ@ z^BcbP^>|3Fb+`EAZ0!DXk;@*%t4P2X31ITI`P;Xs`VWqJ9&o^t(P-cWDGlh>1^s+b z&cQdGg?6rbK$GN6)F#6seK?P9G%Dmfg`6m``{YO~1Le3ZgNOjf%}VDQ$M>FXa#n(} zGMmmQZ9b9tNnUfk%b)Wjy7PaE=P+8W6r#D{{*@*1 zd43-6sIs+Dt4-h6r58hLQ*s^~rtCLQ`?TQN zfT1v}Tx@M?$j1+0b)Rx!{IH}Z<^exM3MG~|6uCT+@+Ds4!zm@R$qq4QV(JzT+u)1h zX0|^Jr6R8C?uMR$@x`!&OE$4l#8JgK#!X%VoHkYSoQdlmh+JFcwf9q&y(;2IU;mOD$ju(RSgEikU zY?#N$>&Ry0L;8IuG00E1^&D(*#l!xCo7qN>64lwo!~~iu{AkwJpt~46iMw%A_d}eW zW3?wy1#$K7vSWoOJmMDU$G`kZT3uZzwkEN6k{iMlm3)eKSD$1)&aX~ZzV#_(rA;&} zD4N9{8pXvHC%mLlV!QL%VX=@n3l@7++!%CGw47#uqO_#eo zw1a(#3+PjNhqc$^Tj|Xm@X3q-iEPPIFB)XvZFTD9*lS?j@&+pBl0bJc5?_G%)OE3z8%>6qz@hFXo}Q>rjMr&_}1 zE>p>!7xESP6Ais=`{v^9xe%9iBptZh&Uk#yB{zhP2Q5NXMXM8g$7v)}^a*sVlm!-+q(9ts9 z4r-`VS}ljaO)rStNs^s@MhDP@jI>K*p+_QNLyYL~{icD#0`D>`a%bC!e}JXs5<8?_ zA3z1dyXnH!9!xyDs|cun-c&M^Zo-UieF5RBXnSA7-cqwDFyJp95^uu%B0csQEbB&e z1maijU!#cHfRTh>cn|0qE@sA#%WB?ESf~`+iJiHuynaJDTSla=FI{YB-1?oaZiHWW z_P)8nnxTExvu-M5@*qk6UYhWi@aFng;cX*+Fpq3&I^JOo37gAst_y-cE|Guv7trRh zD@az`y!LPpxJ~Pf^=#QOxLi`zK!av^=!?!AGT;hC$!DR7^B_;j+p&1}-s)}ZB96+q za+EBVQ_T_yGlwiyJR~)Ia^%hqed(y<{?Q4@(tj_^z3pw9IoP4cYZSMTyKG8a$Z>(F zX5|qw%IM^ZOH@tYQ<5!Ss}URE0w+!AQ2&ZsK)!uU+qo}tz+oXx_SW_JL_cusT;+u+C%SOsHzCax zAI%bVeph6XWw<;+@IFxhGjiPI{_tr)i-~+lijreACRLWc)r~m7qpfc*KYOE=huUt+ z_;xaU%`hR(Y%ynKVm8m!w1tOiVNd`j6Ae~ghl`$mG3X!0T%KRLp+V~_$fESSZG=ti zUw7`cLX!)cFWe-dmv3y<7Q;s@+8Kgb_=MyBiD#i-6S}l=R!@C%sPq%|zJjmJG_mRK zYv>7RMAC!Fm10HiijcRWqU#$ENkQLVvNg34`{=9jvs1NOtd!QdYUS_d_q}ot!=fTt zGT~*Qp}Lc}yV2d|Yr<@Q{8EyC{1Fk1q1wD8pE&@4O(5Wl7)R}=4*Y%v_AQkTjc>l%o_rWRcWN;b4;j*`bdk zKJ7~qeD@wK(*8J8(i;?>JQ zB|Z0u65mF1vn9!Z_+6-REv-EropGlz&4NlJP=P0KMBeGlXZ$hX2`KwII7x0_=%n!M z&BSRz%qA0rSr;+Jx(HO(hc_w40KAiJJ)!&-2FvkxCQ}Y6-P-9!=7pSL2Lp#Msl6Xx z9EZctz*u;86l;0uVcu=5NY@dN?3|44vTU!9AEx5JdMSNorFm-MK>B6*|L9oRRcl5)oAM6j$=RT+g}VLjNl-(LzW z5<=JF=2@zJ{mq4ylU{)Xl`dGBK-^AcY{0rpAYd4>yK5Z`1&cuMzG7@FYV7U;a#3hd z9zz^s;g({*S#J7FlS%JGHQMao)XIm`KBLKSVZ1>uQgO7!^3Uzr9>WTU zSzMSdrB6pFM4gME@0lhIw?wd-_%eOI6ms8t=H22QEs25At7G`acL5AxHo{pao~v4G zBbZC_N?l`Nzki((-Wn3{YOe;{0dX+al9(#EkjmQi^B$24snut~FatKY>FZ!Y zR=?~y8eVw2ddg$6f^wZ1YuGmhs1d@#7dLoI8Rq_xt1nJ0lY43FkMh&txf@SdEDlTz zE_+TV=wm(i)25@{v8ht61gh&<3q(Ae8B6RO)k!Nf&gqSO)SBj>Tt1!uActa+dqIit znP?&V+}3v?2Z45|GUW%&4?ly1oOvQhxN%1u9F*zIjOCJQL2ZjH<&>Mxr`iW?%^qTR zmRD}IrK-PLnPw}o%vP0u@n@ydihledllG|`#7V=%m(s5!TU}ql(=Qa@(pgl1TEA=A zUXLmOyg?D(WxnEKCe-=r&vCday04)xm<3Z@smFB}e2|?^yGY#YViUv6QVmQ_hlC|x zcZEx^Mg%NoFL-j_oU_s)EaDSYbaGEPeY1xc$L*t0XKH-c&HMMVvbrDZkz6?Jcif8= zW6hZuMFH`_*i>zpV~Kzpfev@RbbcH+lvzEMZS-_D;dcN93K)jctn)oJ_wI?ajq7t4 zsBa4qAMAcPW|Dzh0(e5EtfD(F%nMlFh{6*qbi7x6qRKYR-c(k_|4gcay&VXb=I8ie z;1qTG8mSQMn%=twg4`~7pi7^lNN4uZ=vjhJ^;Cvw@NmXSR-x4dK@cngWI~o1Lf%SG z4nT(^N}-abtY|;YwH3b1WL9-XPe*IzoXZA+E+Z1PJ{s#J$%J%m%MC7XL=G#6)zb5S zt&oq{CH)A@TsXNi_ceiGc{lgGC9BxIq5($TZ#OsT15u%HJ!a+w+nhkk2rmW%VY`_2V&i(?45!aI` zJtf2pQv=&dn`T>SK{cwNti3^9y9^*{rO5l*K})^WtRr;Lfcw%urygv<>4-5FGszUM z54P$h1?a9{G`oxYg`r@ePC#j<tzxV!9>+pa+e&h>UwNgN@;%7t)AJo_AP}Llogs~ z4N?x*wl)zTUJM#T)*HKS)e^G^pj^>&vkRo2*sGsMlF(jVXPgDCL7~9UpxA5iV6x&( z(^7}3CTUg-MQhxjNJKm{$+hZoWUyz+P{FqZzN!cDQd)YhxEn>%LekEyeGg0}Hdm-D?<;HF z5#UZEcb6cM%Z0P%_;;m9jCF7>`x8S~SEoNAx=!Vup>3NJ0Ec;1;p#8_6lIf-nY&bD z7h5&)Uk%K!^dl+UJKIU3S~AtlOm{+OmP#@R=rR?@OtjUD5A=AqEIU7+EPmg&kn|hR zlvIU}URb6pz*G^9&yH^~ZP&qn7wv+z z>qm{cG0!O*YmTIF3IK^cxI0uFY6fNc|2lo`=bP)=Pi;Z%o(V@?;St<>)V=#zk#i=A z#pQwiyw!5}V8i7m&V;) za|4^UO1ZAYfr8ID*k}uK2A9jE8)Nb0#0nlP?EFHso#yjRUqQA~AN7n$)*tY2l#YM>b zC3}eXwWg=gLJx-<6~<(muQ{_HQgG3Pa%qCJRs6W<{L*EO;3JjK^FcxIj)X>6UiEd% zsH?083NuW;XYHsyODQ8&?~F7`s0!K}ymK$HmtK?xeucdU>_%P%dfcFkk9hQekAfn+_Cf5K^Og17LaD?@$TS|F4Vwa(*a0 z1^mu(wGGO;ZLVkB4=@nv)2qb*V9_zh`$8KB9M{_S9cfCr0`gb; zD}=wBLdpB+vqRq=MY!c`Y_nSdNpDUJIo4Y)L9{`8$~ey(f>uvgmHRh~@OYWuHox6u zJ+l8SP`Y8GU!2}RqXg8*AS~4DrqHABOg!rxH5`8Li3i1!qSZ_JTbm6}V2|TPZ^r+E&ESJvAKHBLPp$nT-oOxj zDgzX}NxCWu`$wD$eiRE)Jw4ch-1Ej!%-2B6eP$xMLHj}Dj$#-l(+k~P?LoUUh4^IE z7>9Y@8e3cbyvnY^*+21`mlTaskM4E{eW@{q*;H4Kk}pU8Wg9rNq4rYa6ur9d)q|%l z*zUIPYq?--qMc`y3AbyZd}X!Qo=SY-EO)+4T|Q%lOvFU?NJ_e` zF3&Kwk_dKu2Trg)4@a%_&r$QrfhgQvPAyH5b~l{*5nZofgi5PB%6zYeVe?B!xM(XYJ*}W6J*6+aJ~GLhZvlCjFlu|4VQo?%hA2#`xd~hk13bQhjTy81a^wR z#-XEP-MAR+$JZo{zDS-p zV%0q1A%U4{E+5u)7Es?YmvgRz|W z;z?V&$R$18(bJ1-(PlIKr8i|Ai(GR{zZJP&R}b$GS(*24P$<)1pF$^2FdNtg7vk_twn1CFm0%Qs$M2 zVjs>@sKgjUxf^9z^Xx=4c$-89kg&-^tFvkoOWp2ws52^FR=x2j6)H88nYZJQsm&ou t*p{T0>p?9bVn5WH{eSeH6TOLDjQQy6Tcd)K;H52WCI*)Jm6zQf{2OU1p05A^ literal 0 HcmV?d00001 diff --git a/img/structure-spatial.png b/img/structure-spatial.png deleted file mode 100644 index 0e898e3e18d7dc0fa89e9685d222c95a5c9699a7..0000000000000000000000000000000000000000 GIT binary patch literal 0 HcmV?d00001 literal 8209 zcmcI}XIN8P*X>4)C{Yjva|9s>Qlu&v5D=r%n;^XxDbjl}bWssOq=QO30s%soUJOm8 z7YV%wDKYdGkP^5%dfxM%`+e{C-23C+A6a?!de++e*>lb@=2&CDQhy+SmWGK20D!ZK z3Nji1KqUzNtWKQ-XSAmvOW+@syN3LIps@Qg5xh9@_@3%L0QeXQJuruW*QZ?+^xXmA zya(k+HE9ih0s!zVMVWhAPfV64R2}Sz#x3iC)>(Z|FY&NxT@9(@U+RdmeySy-ogE%V z^XisHbjeBV7vZu33bUoGM6nL{U!Jz@ z1W+?wJsmGij;h)*lJS~$n$uR)&I@|R7EMq|9@939+MXC81wpr2|MAO;}3Zn$QJAwYTxY zjhXVAK9;0LwzAHd!vygv!0g6T4namO>w9-j3sw@5PR^g9K!Ol|)ok^IiVI3pCQxMP zMX{zF`DDZ3U}5It0CKFcx6=m7RM_(}s(ach2k0-@4@ZZf40~ z?s&AwK6Z4V@Fs1@y^P~IPfVgy_w~J(S!@~Xh3)n8$9USGCXXzx`rzV17*RKhC7cq#>dqsgJnebmg}l&jtbUrWUtA1jb9161<7~i{18tW5aUUzs|kg? zjyszN%N{)`C)aqMM^tq#pA*6??Gb1Fgy{sutHZ`Ll;lh=c{nf0xHr5W_p)>_qh@0= zw>Pr(y)t#w9f-lg)%TvXJ&ot-pHmPTjO1_LnadO<7&qyR)MCmx%H5Qex}?pcPJZF@ zi%?GO=m|I|^jGQY6Ll}PIute8E0)xIgViKIuuyKFWE$GtK6LpQ5Uf!uFxJvWva*ff zA3PrVa62Jw^(d*;_viy{f6k&spd;M(&ncPYL@$YRsRZ=&YDnIZUCFp$g_?Uv8<}Z=z;fqKP3?~##GPWRB5R{Y@}o3HgB|| zG50iY-sAmA+H%Sp#|`vr^b^LDaS!$-Cf&z9N(0$zR~8EjMjVHL@8^Hlm!T=7Ua-B& zUe|N~MFD=NouM{jrqSJ}D`t=J{^a-^}8~i9}|{YJ?z*y2N!A6%2OI zzRJ`Na&%@OqtTs)#PM)Fi&ZIi52}IGh%}li5k2V;T1-D=W@rH}u|Vu=#C&`SIJ^!* z6kaWhp$s9ur9>7W|L$;T$2jfBd71#grD?EgBL}a>UosCd%Xm@b+P25{(`xlwo%D|h zS}Oed$PaK)(zX52aPVIo>RfV&f5~3t_k|K_X)ts@PK>uefL-<~K;|X>*g2M9=v|Lf zfv=Qy4T!e=gHH#UCnQ*fY>;h#= zjaI>mky%@yR zPA^-tgX?``7e1~SDRiW<+P$bTB9ep?QBm3P zONuvvBsLITZaH{erLGDGICrIF9C}V@5LP+C@X%RSyl87Su2%PH#IO8V!d)p*eyp{2 zJd~DF^A}v;Fkm+Pe@qr7dmn<~cRhq7tH8uXRR&?$!F?Ny=Zt(TDZxKDaWZkVASXD` z3-(CfZcqA{Z4?tcp*YA#B2Eu@2)*oz`P1MbBv28e zS-I|TUsBCfu%~wAj|Pj27mc#c>A{|v5ovm8YU;@SpKTxV_Dd!M^4L@Nb2=h|ka z&LtGqC!{04_D2joTO4Q(JrkG54V4)V zwJ$En4GvUG)p;4dMecAX>ZFaX$fR9+wwz0!AhtZlt7P#MT^vnBr#?6qD%ZfId)}ap zGJM94Y^|_v(=RF*lE(`vWTQ*_8R!Q^^+fE4UC8C*JsY2y`1NS@4Ay{l% zaYb3MIldAkCaR`NfU`PbjNkp!VS!HwO>N%vR!N}m@*Cg_n?r=X0ONtzGG?G8d?VIP znZ{FGL$lrT#^geWq5w(Id`He_<=aM6VIVtoc&7l>?LVgNjY>jj4zbyG4mWKd1}N0K z4;NNsE)}?@IQ!lsTwLrzk-I<8)QU(!RR_P)%5=ZwJ@)Wdy_Ec}7ZDKUGA}{4=sjH( z2?aJ}|EWCwX5W8F5p}ROCEm#!f#un1*q{{+?%Sm~B`8&r4whiRv1iK5I^Yr@yp`36 z>+P0}Qe?gkSk6ExghIez5yFA_(@;UpQ+ZI41jEK6zD;kny^NHX7Z%p#`E~~6poq1c z6Do|cy$O>Erj1jM40Ct8J9pk|MLP5DFiV;wrq|5Zs63&HRJiL>-J4)IgQK!K_9b~U zSb0^N_h+7CHB;FhLuSBWoW`PF|6LW=_M^bzBN#ETRXz-!%frL4SAka$xuQdF(y!@J= z@Vh1JFaB_*(v+ppHy=g8g;BC}`ix+Ccz*Dd#|;hhlMc`SjsIt0k1T&>RR(OpZv9uj zUM#|!mSvfm64KN>sfpDYOI!rasn547$8qJOVB3tjHvgIc=%oc=9tn@)#=urojvD%b zzh)|ie%zh6{6nriqEOPFB;=fla|bs{!`#Np^J=O;2SjR)!EY{#1B2Xuff+Mpwjb{B z3(a6}@t*JBGA&aA)mENJ_ax@Wi6}q~BtVRsu%>jD?E`yrR#Q5?4EZ;?tS6GnlCXO&Q^w8E|l~z>>LY7I^`|$28 zztr-WWSlj|D(}!!Nf3MydwOHY5&IA@>;S4I3bOPYYQtz44*)kAN=pBqs3Ru`;{k_p zXqy}v78LjIcpmU4V3E4zZ!T) z0Y1%+&{jOo*pLDHOC5nt^;BOb2k!P|;!mozR!}KWG*IEyVilELke#nE9u#c2n45?Z z$Bj|tJHR;!Fl!y4fDr7iM2|YQicD&qh&o@~9RFOWq#2RS&IO8|3G7mvZEfyJGsKlz z;3tUNd9Tm`t+hxObbiY?EmgyXistfmnnQXB2Cj`ByLGMvdY2t+u{xTK$mZ8IHMm>1 zrcDHa=4ZKh>LA&FwbW_VLHL3CYoF5HqGrnH(v5c1xBl)UUUA*geEGw+d zM%lR|4?iWT-Q%n)tE5UKu6o+p?|9lU_V8`rz?OSwKJ+jSCtny5FjX#1OwM~tP&qKs zEQFj(3&Oi)6liO&MgC|l4N=(47kqXxV5t1bjX$E6TVrQuP5d7i2PW84-PSVaC~T|< zrU@`5+&92!bL3n4WLKu&tXc6hH|6F1(!Sv8Y%Q)&S!qLMZ=`6$Ng2-WSohVUbCkx& zX97z_+JgIw`iHPtMv0y|UAc=rZUR9s`5nHL7t$PluZWv9a3Tn#!xr2;hYa%zksaj# zYi2Q`P+Mos+c}d!>)Co)o21nXXCcjH&ad<=^fzbuOFon3|9WJy3BE@Jg{<#E>J6U^ zEvmN(oY`Y(@*0quAFS%G+*y8vWr`lLP4oSPbJtYhBx9|Op=`V(N6{pUTs#4;f1FN3 z@5)l-q2{b_B753T=ADKRpVhR9#^a>s#Br*%DiA$#U@DWJQ-f0OD*g^g<*@&e$}Q_6 z7n!GihV1ISgW1*ken38I4)7@59hKT{lG+}%FKT-nup_cDXjo57Jqzq4idWSgl8QQ3 z{imTD_Hc5}uFu-+teNs`%WA#Q)WN*iTxl9P-*3@ScY`>!TXryL=v)m2@BzYVjHFhj zv)Qw|;joQ^>d92P3cB$-ZB0X_fyQ@ss*O>E6z|#EDRLp|46wrI?n1=;DOD`_8M*DZ zY;PJ{ITc_Su=Whzsus8>vcm(6tAQu{w69j2X$@#jd7m=ULLNK;xi^E1uK(FlALE0) zBBU~`swf7u@_>ZT?sr~BWV1E3Y)HtdO#Y$sXSKVClvgX#C%e{LJtsvsT(kYfN3$`Kf!0MY!Au$pqNBV6j(&g3kOAZYT z%#s<9OOERctsyFb>9ul0JWAzKes7M}v&hQgi7|0C1UKS~)}|hKFW3cgH_8_sR8s3R zIv#T~x_F>nud~_#O^}GHlTT+w`BGFCV52O=F^=Y*$i=kF3ep~I$?XabODU6!773Cy z$sgUUk@bfoWI6E^1AT)Uo9*^RAFY${8!1e(@$XeJUd(QHZ%`NCycfQ?9ogfL_IlQW zg{lm9p*u`#HKdr%KHpz#mCd^vPYzyU5k>dcyuHqA5O-)|XC^v_7+noae$TYhfnnYXK|uJ-QjIZH@Ol9L#!gUod-r@9AFbe z*Iw+=LrD$iPvpB2PvNdvWd{)(a0lOTofF%{a9rnJA9;$zywEX+0#8e+zA2vzhWHhC z4utnxxorniH<(DZi4zOa{n>T3#bSCn^&XKxtDsq~4UE7&9|a95yEojW6}6{rm|L*% zvn%<+!X^Q?2E z9LiT({Ss%L!i<)?>Mdm(R`;$iX_Cdc!@Sv>x$6fvO6*40>y@i6(UaI_dqxv5&f#?^ zgy+!+gY}0=mgB90n=PY$b9tumLILY^b6sMA6H>xcQ%9eX?Tv|bSJMt=xyLzxbAs+J z2(>!lKg>Ddf7rm5V(BhuMPiDUT}!Z0Nb*Lawu?W`9|oL~k41oj&Sd6T&?ox;>It6v z=fvNK#aL9X&7E{PM~aA0p2rKrX>4ehy4ND5zzMwObFQdZx*4_rbGwfv50~q%Y)(4H*ORqy z)4SW-1v?#eW)nxPJ;m^`+t(EQ4+-Nt9;Ir%A*bVy8r)9&1A3h-0tSu9cl$ZB8X@i7k$}%2oS}quuab)PAHum&fCIXLSXqkjX zV+_A^#|GtOY{1|0IOP7|jvqUlDXn~TW9W@he9A7TYA?cE^I|@f#8+JkRhb|E@$k z*FY!GHSU;J#}oUezrA4Xt4Eh99ZkXhl;q7-C)OgqQ@?h+my0YxFf8wtcMp#$vGJJTt=Mr_MrZDe{ zT9A+dQ6)_ax>+Hch>qt`haKqN>v8?j#am+m?EdT?GL@{xvYMQyEK5^YfV=_KAg#W}Cz#BaF3 z&T0BRhJHC*GE;%dan`9(K^NN8AmGs)0xj2*soCoZ{=UV zca7V}JWs2pbY(dDeJQd-%2H!sfwlqynBST{H^C#BV5r59<%Z{diL!{&sv!+j1B( z`-b8({Bj?DdyMu*K$O^j^dSE>CI8+5DwZz!{+tMK0@ZfeXkMDC>*>U$!;cj2A}>m+ zYK7u{ftIja_?qImu1*eLoC(r2p`HApE_o@J5K+FRS6Y{iFeyNmrcZsG8XEgVh)hZP zx%s$$<;zd|OBHTH8xl`d8OoZi?kgP4;cWg;T4+6d%t;t<{}evos$N}05}Gqo(pX6C znCs{Eobzi}G%g4ZK0`T6d9%_0)tV;ieeow#|kj#;fNRv@#7hodWY z+*d}~VF3YZ{wbi{$mDwVSev7c#Jhf>Xvubs#58V0Ky4n|)W*{ufWA41>#Md=#FP|uTYuz! zsY=4O83`P>c2VENS z#BXlqR!#<8b|!Omr&%Q@lo+=8M4Fr^#@(XIZ{zMwA$)gUdn<3q8x0rhGhj5s^nO5f zzEIp9HOy;2s3E-Fk$8HRp`sZPBQ03G)h(4yG);fLyYqVGR=9rrKG%~i$tYRpvWK8x z$-+L%k#~JG*u9nh>9s&H^}{{`sXZm?$fLq^I2RPV^82JA9en1=iDQFR9w&%#ZuN0{ z-%^>@{z8vIYgxPhz7t;IR@HCq;dkOQj)E| zL?PxR8OR40+XbQ^xpG`r^ zjtBFY^xaUVa>^})-aYH^qzSwfW<$asOFk$a}BT99|=B`l{vaWyol9=J{7+b!tDh z`|dv(@LSnLq~pHd20Ls%h=Z5IJkZO+sy9lcos`LTD3sLoS#vxH{xW ztAeFwkyCv&i;okLFWlFX#`pB9a>{DJZ!3qg~bZ25kG-tJ? z{Nkbw;%E}>sBNYRL3q&w?Z{}Qse&1z@Fe(=>YfC8oJSr$s(CyB>=91;Te8r1I*s*j zzgHBN2Xggv*JkwH6N~W68p6&a3`))DZr`HP{7Mb71Dm1!RE&;Wt#p$rhhbZd?)N)j z$OpOpDGM$}7vkY>b&;!f%LB!{+P==&(RPPCY6mT=55cJaa$i{fcAnclHA4Ut$3MS8 j{muV>efY4HIfVB;rav|#u0Rid#s?H-AIKEmHw*q>AL$pB diff --git a/08-testing_in_R.Rmd b/testing_in_r.qmd similarity index 86% rename from 08-testing_in_R.Rmd rename to testing_in_r.qmd index 1c752f20..aa812f25 100644 --- a/08-testing_in_R.Rmd +++ b/testing_in_r.qmd @@ -10,9 +10,10 @@ If the new code requires GUI interaction or accesses the Internet, then it is es ## Writing tests for R -Writing tests for R is much like writing tests for your own code. Tests need to be thorough, fast, isolated, consistently repeatable, and as simple as possible. When you are adding tests to an existing test file, it is also recommended that you study the other tests in that file; it will teach you which precautions you have to take to make your tests robust and portable. +Writing tests for R is much like writing tests for your own code. Tests need to be thorough, fast, isolated, consistently repeatable, and as simple as possible. When you are adding tests to an existing test file, it is also recommended that you study the other tests in that file; it will give you an idea of which precautions you have to take to make your tests robust and portable. Tests for base R live in the [`tests` directory of the R sources](https://svn.r-project.org/R/trunk/tests/). Contributors mainly contribute to the "regression tests" which are in the files `reg-tests-1a.R`, `reg-tests-1b.R`, etc. These tests are added after fixing a bug, to ensure that future changes do not cause a regression that reintroduces the bug. If you have been asked to add tests for base functionality (something that is not in one of the add-on packages in the base distribution), you should add it to the latest regression test file, with the highest numbering (currently `reg-tests-3.R`). + ## Test files Most tests in base R are `.R` files with R code implementing one of two types of test: @@ -20,7 +21,7 @@ Most tests in base R are `.R` files with R code implementing one of two types of 1. Tests that produce an error on failure. These often involve a call to `stopifnot()`. 2. Tests that produce output that is compared to reference output. -If a test file `.R` includes tests that produce output for comparison, running the tests will create an `.Rout` file that should be compared to an `.Rout.save` file with the same name. The tests are usually run as part of the check commands `make check` (for a basic check of R) and `make check-all` or `make check-devel` (for a thorough check with/without the recommended packages). Alternatively you can run a test file directly, for example if you have set the environment variable `TOP_SRCDIR` as the path for the top source directory: +If a test file `.R` includes tests that produce output for comparison, running the tests will create an `.Rout` file that should be compared to an `.Rout.save` file with the same name. The tests are usually run as part of the check commands `make check` (for a basic check of R) and `make check-all` or `make check-devel` (for a thorough check with/without the recommended packages). Alternatively, you can run a test file directly, for example if you have set the environment variable `TOP_SRCDIR` as the path for the top source directory: ```sh cd $TOP_SRCDIR/tests @@ -34,15 +35,14 @@ mv reg-tests-3.Rout reg-tests-3.Rout.save ``` Note that only substantive changes are important - the check will not fail due to differences in the version of R, the locale, or output from `proc.time()`, so if these are the only differences the `.Rout.save` does not need to be updated. -``` Sometimes test `.R` files are generated from a `.Rin` file, for example `no-segfault.Rin` generates tests for functions in the base add-on packages, calling each function with a range of inputs (character, logical, `NULL`, etc). -## Tests of add-on packages +## Tests of base R packages -The check commands (`make check` etc) run `R CMD check` on the add-on packages in the base distribution, so will run any package-specific tests. If a package has tests, they will be in the `tests` directory of the sources for that package. +The check commands (`make check` etc) run `R CMD check` on the add-on packages in the base distribution, so they also will run any package-specific tests. If a package has tests, they will be in the `tests` directory of the sources for that package. Here is an example of the structure of the [R package `methods`](https://github.com/r-devel/r-svn/tree/main/src/library/methods) with the `tests` directory highlighted with yellow colour: -![Screenshot of the structure of the R package `spatial` with the `tests` directory highlighted with yellow colour](img/structure-spatial.png) +![Screenshot of the structure of the R package `methods` with the `tests` directory highlighted with yellow colour](img/structure-methods.png) The base add-on packages use the same form of tests that are used for base R as described in the [Test files]{#test-files} section. diff --git a/testing_pre_release_r_versions.qmd b/testing_pre_release_r_versions.qmd index 31f7942d..56dae8e6 100644 --- a/testing_pre_release_r_versions.qmd +++ b/testing_pre_release_r_versions.qmd @@ -1,6 +1,6 @@ -# Testing Pre-release R Versions {#TestRVer} +# Testing Pre-release R Versions -This chapter is inspired from the blog on [testing R before release](https://blog.r-project.org/2021/04/28/r-can-use-your-help-testing-r-before-release/index.html) and discusses how you can help with testing of pre-release versions of R. +This chapter is inspired from the blog on [testing R before release](https://developer.r-project.org/Blog/public/2021/04/28/r-can-use-your-help-testing-r-before-release/index.html) and discusses how you can help with testing of pre-release versions of R. ## Where to test? @@ -8,7 +8,7 @@ Whenever possible use a fresh package library for testing, even better would be ### Virtual machine {#VM} -A free Windows 10 virtual machine is provided by Microsoft (with a 90-day limit) for building, testing, and checking R packages and R itself. Package maintainers who work on Linux and MacOS can use it to test their packages on Windows. Read the [instructions](https://svn.r-project.org/R-dev-web/trunk/WindowsBuilds/winutf8/ucrt3/vm.html) on how to automatically set up the machine to check R packages. Tomas Kalibera describes the details of using virtual machine in the blog [Virtual Windows machine for checking R packages](https://blog.r-project.org/2021/03/18/virtual-windows-machine-for-checking-r-packages/index.html). +A free Windows 10 virtual machine is provided by Microsoft (with a 90-day limit) for building, testing, and checking R packages and R itself. Package maintainers who work on Linux and MacOS can use it to test their packages on Windows. Read the [instructions](https://svn.r-project.org/R-dev-web/trunk/WindowsBuilds/winutf8/ucrt3/vm.html) on how to automatically set up the machine to check R packages. Tomas Kalibera describes the details of using virtual machine in the blog [Virtual Windows machine for checking R packages](https://developer.r-project.org/Blog/public/2021/03/18/virtual-windows-machine-for-checking-r-packages/index.html). ## What can you test? @@ -26,20 +26,10 @@ You can test: Details of performing testing on various operating systems: - * [Windows](https://blog.r-project.org/2021/04/28/r-can-use-your-help-testing-r-before-release/index.html#on-windows) + * [Windows](https://developer.r-project.org/Blog/public/2021/04/28/r-can-use-your-help-testing-r-before-release/index.html#on-windows) - * [macOS](https://blog.r-project.org/2021/04/28/r-can-use-your-help-testing-r-before-release/index.html#on-macos) + * [macOS](https://developer.r-project.org/Blog/public/2021/04/28/r-can-use-your-help-testing-r-before-release/index.html#on-macos) - * [Linux](https://blog.r-project.org/2021/04/28/r-can-use-your-help-testing-r-before-release/index.html#on-linux) + * [Linux](https://developer.r-project.org/Blog/public/2021/04/28/r-can-use-your-help-testing-r-before-release/index.html#on-linux) - * [Solaris](https://blog.r-project.org/2021/04/28/r-can-use-your-help-testing-r-before-release/index.html#on-solaris) - -## Writing tests for R - -Writing tests for R is much like writing tests for your own code. Tests need to be thorough, fast, isolated, consistently repeatable, and as simple as possible. - -When you are adding tests to an existing test file, it is also recommended that you study the other tests in that file; it will teach you which precautions you have to take to make your tests robust and portable. We try to have tests both for normal behaviour and for error conditions. Tests live in the `tests` directory. - -## Benchmarks - -Benchmarking is useful to test that a change does not degrade performance. + * [Solaris](https://developer.r-project.org/Blog/public/2021/04/28/r-can-use-your-help-testing-r-before-release/index.html#on-solaris) From e85618c2c2ec4a462940e83be883c8750038fb29 Mon Sep 17 00:00:00 2001 From: Saranjeet Kaur Date: Thu, 5 Dec 2024 16:34:29 +0000 Subject: [PATCH 09/11] add chapter to _quarto.yml --- _quarto.yml | 15 ++++++++------- 1 file changed, 8 insertions(+), 7 deletions(-) diff --git a/_quarto.yml b/_quarto.yml index 561939f2..d8002a22 100644 --- a/_quarto.yml +++ b/_quarto.yml @@ -17,16 +17,17 @@ book: - issue_tracking.qmd - reporting_bugs.qmd - submitting_feature_requests.qmd - - reviewing_bugs.qmd - - finding_the_source.qmd + - reviewing_bugs.qmd + - finding_the_source.qmd - lifecycle_of_a_patch.qmd - - documenting.qmd - - message_translations.qmd + - documenting.qmd + - message_translations.qmd + - testing_in_r.qmd - testing_pre_release_r_versions.qmd - r_core_developers.qmd - - where_to_get_help.qmd - - news_and_announcements.qmd - - developer_tools.qmd + - where_to_get_help.qmd + - news_and_announcements.qmd + - developer_tools.qmd format: html: From 4d6344e2a1c87954925ae99154dda3e8aa27f93c Mon Sep 17 00:00:00 2001 From: Saranjeet Kaur Date: Thu, 5 Dec 2024 17:08:04 +0000 Subject: [PATCH 10/11] update quarto.yml --- _quarto.yml | 47 +++++++++++++++++++++++++++-------------------- 1 file changed, 27 insertions(+), 20 deletions(-) diff --git a/_quarto.yml b/_quarto.yml index d8002a22..1f5bc17e 100644 --- a/_quarto.yml +++ b/_quarto.yml @@ -10,34 +10,41 @@ book: repo-url: https://github.com/r-devel/rdevguide repo-branch: main repo-actions: [edit, issue] + sharing: [linkedin] + open-graph: true + image: img/cover_page_scriberia.png + downloads: [pdf, epub] chapters: - index.qmd - - introduction.qmd - - getting_started.qmd - - issue_tracking.qmd - - reporting_bugs.qmd - - submitting_feature_requests.qmd - - reviewing_bugs.qmd - - finding_the_source.qmd - - lifecycle_of_a_patch.qmd - - documenting.qmd - - message_translations.qmd - - testing_in_r.qmd - - testing_pre_release_r_versions.qmd - - r_core_developers.qmd - - where_to_get_help.qmd - - news_and_announcements.qmd - - developer_tools.qmd + - chapters/introduction.qmd + - chapters/getting_started.qmd + - chapters/issue_tracking.qmd + - chapters/reporting_bugs.qmd + - chapters/submitting_feature_requests.qmd + - chapters/reviewing_bugs.qmd + - chapters/finding_the_source.qmd + - chapters/lifecycle_of_a_patch.qmd + - chapters/documenting.qmd + - chapters/message_translations.qmd + - chapters/testing_in_r.qmd + - chapters/testing_pre_release_r_versions.qmd + - chapters/where_to_get_help.qmd + - chapters/news_and_announcements.qmd + - chapters/developer_tools.qmd + - chapters/additional_resources.qmd + - chapters/r_core_developers.qmd format: html: - theme: cosmo - include-in-header: header.html + theme: + light: cosmo + dark: [cosmo, css/theme-dark.scss] + include-in-header: html/header.html description-meta: "This guide is a comprehensive resource for contributors (from all levels, beginners, intermediate, and seasoned) to the R project, facilitating their onboarding and helping them navigate the contribution process." pdf: documentclass: scrreprt - include-in-header: preamble.tex - include-before-body: frontpage.tex + include-in-header: tex/preamble.tex + include-before-body: tex/frontpage.tex epub: cover-image: img/cover_page_scriberia.png From 58199e5146a81795b68cb28b17aa5d4ab471fa49 Mon Sep 17 00:00:00 2001 From: Saranjeet Kaur Date: Thu, 5 Dec 2024 17:27:26 +0000 Subject: [PATCH 11/11] move testing_in_r to chapters --- testing_in_r.qmd => chapters/testing_in_r.qmd | 0 1 file changed, 0 insertions(+), 0 deletions(-) rename testing_in_r.qmd => chapters/testing_in_r.qmd (100%) diff --git a/testing_in_r.qmd b/chapters/testing_in_r.qmd similarity index 100% rename from testing_in_r.qmd rename to chapters/testing_in_r.qmd