-
Notifications
You must be signed in to change notification settings - Fork 0
/
Copy pathsearch.json
4855 lines (3378 loc) · 219 KB
/
search.json
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
[
{
"title" : "UK Lustre Users Group",
"url" : "/talks/2023/12/computational-storage/",
"tag" : "HPC, exascale, storage, excalibur",
"category" : "talks",
"date" : "2023-12-06",
"excerpt" : "In this keynote talk I introduced the work we have been doing on active storage (also known as computational storage) as part of the ExcaliData projects within Excalibur (the UK exascale software readiness programme)."
}
,{
"title" : "Using Deep Learning to find cyclones in model output",
"url" : "/science/2023/07/tcdetect/",
"tag" : "aiml, cyclones, daniel",
"category" : "science",
"date" : "2023-07-09",
"excerpt" : " Galea, D., Kunkel, J., & Lawrence, B. N. (2023). TCDetect: A New Method of Detecting the Presence of Tropical Cyclones Using Deep Learning. Artificial Intelligence for the Earth Systems, 2(3), e220045.https://doi.org/10.1175/AIES-D-22-0045.1"
}
,{
"title" : "NCAS@Reading Weekly Seminar",
"url" : "/talks/2023/06/what-lies-beneath/",
"tag" : "HPC, exascale, hrcm, modelling_futures",
"category" : "talks",
"date" : "2023-06-23",
"excerpt" : "In this seminar I laid out the background to the new ENES infrastructure strategy, including not only the computing context but a good part of the science motivation for a spectrumof climate modelling activities. Below I sketch out a summary of a few of the points I made."
}
,{
"title" : "Draft ENES Infrastructure Strategy Open for Comment",
"url" : "/science/2023/06/infrastructure-strategy/",
"tag" : "exacale, hpc, climate, modelling_futures",
"category" : "science",
"date" : "2023-06-20",
"excerpt" : "One of the many reasons I have been so quiet of late is that I have been bevering away on a new ENES1 infrastructure strategy for climate modelling. We now have a draft out for public feedback, so please do comment! ENES is the European Network for Earth System modelling - main website is at https://portal.enes.org. (Footnote added, 21/06/23). ↩ "
}
,{
"title" : "Dual Tariffs for EVs Don't Play Nicely with Heatpumps",
"url" : "/ecolife/2023/06/dual-tariffs-4evs/",
"tag" : "ev",
"category" : "ecolife",
"date" : "2023-06-18",
"excerpt" : "So we have an EV (a “2021 Vauxhall corsa e”, since you asked). We had to get one in a hurry as our 2008 Seat Ibiza ecomotive died very suddenly and unexpectedly. We had always planned to replace with an EV, but imagined that would be in a year or two. As we live in place with no public transport, and my wife and I both have o(20) mile commutes (in diffiferent directions), we had to buy a replacement in a hurry - without doing much due diligence, either around car or charging options."
}
,{
"title" : "UKNCSP Strategic Workshop on High Resolution Climate Modelling",
"url" : "/talks/2023/06/data-challenges-4hrcm/",
"tag" : "HPC, exascale, hrcm",
"category" : "talks",
"date" : "2023-06-16",
"excerpt" : "This was an inaugural meeting of the UK high resolutionclimate modelling community under the auspices of the newUK National Climate Science Partnership."
}
,{
"title" : "Baljifest",
"url" : "/talks/2023/05/perspectives4baljifest/",
"tag" : "HPC, exascale, modelling_futures",
"category" : "talks",
"date" : "2023-05-15",
"excerpt" : "Baljifest was a celebration of the contributions of Balaji to climate modelling at GFDL and Princeton. There was a day of presentations, followed by a panel on the future of climate modelling. Saravanan and I kicked off the panel with a presentation each."
}
,{
"title" : "UK Turbulence Consortium, 2023 Annual Meeting",
"url" : "/talks/2023/03/data-for-uktc/",
"tag" : "HPC, exascale, esiwace, excalidata, cmip6, ipcc",
"category" : "talks",
"date" : "2023-03-27",
"excerpt" : "An invited talk to the UK turbulence consortium which aimed todescribe some of the activities the climate science commmunityhas undertaken, and is undertaking, to deal with the data deluge."
}
,{
"title" : "Two new preprints",
"url" : "/science/2021/10/preprints/",
"tag" : "climate,, epidemiology",
"category" : "science",
"date" : "2021-10-05",
"excerpt" : "I’ve just added links to two preprints to my publications page. Quite different topics. (There are also a lot of recent talks online as well, I’ve had a busy month!)"
}
,{
"title" : "ESiWACE2 2021 General Assembly",
"url" : "/talks/2021/09/esiwace2-ga-wp4/",
"tag" : "HPC, exascale, esiwace",
"category" : "talks",
"date" : "2021-09-27",
"excerpt" : "An update on progress in the ESiWACE2 fourth work package, WP4, data (systems) at scale."
}
,{
"title" : "19th Workshop on high performance computing in meteorology",
"url" : "/talks/2021/09/digital-twins/",
"tag" : "HPC, exascale, digital-twins",
"category" : "talks",
"date" : "2021-09-20",
"excerpt" : "Presentation: pdf (0.3MB)"
}
,{
"title" : "ESCAPE-2 Final Dissemination Workshop",
"url" : "/talks/2021/09/excalibur4escape/",
"tag" : "HPC, exascale, esiwace, excalidata",
"category" : "talks",
"date" : "2021-09-03",
"excerpt" : "ExCALIBUR for ESCAPE"
}
,{
"title" : "Benchmarking for ExCALIBUR Update",
"url" : "/talks/2021/09/esi-to-exc/",
"tag" : "HPC, exascale, esiwace, excalidata",
"category" : "talks",
"date" : "2021-09-02",
"excerpt" : "From ESiWACE to ExCALIData"
}
,{
"title" : "Under the hood of a Digital Twin ESM",
"url" : "/computing/2021/06/under-the-hood/",
"tag" : "climate",
"category" : "computing",
"date" : "2021-06-16",
"excerpt" : "In which we peer under the hood of the idea that ephemeral data produced by a digital twin can be used by collaborators who co-designed the digital twin experiment."
}
,{
"title" : "Digital Earths - The fourth phase of (ESM) modelling maturity?",
"url" : "/computing/2021/05/modelling-maturity/",
"tag" : "climate",
"category" : "computing",
"date" : "2021-05-26",
"excerpt" : "I recently introduced my definitions of “Digital Twins” and “Digital Earths” and concluded with the statement that I thought there was scope to do good thing in (these new) twinning activities. There are two such axes of scope: scientific and technical. Here I want to discuss the latter (and it’s impact on some science). In the discussion which follows I’m talking about Earth System Models (ESMs), but the arguments could apply to many other modelling activities (e.g. epidemiological modelling)."
}
,{
"title" : "What is a digital twin?",
"url" : "/computing/2021/05/what-is-a-digital-twin/",
"tag" : "modelling_futures, climate",
"category" : "computing",
"date" : "2021-05-22",
"excerpt" : "There is a lot of talk about “digital twins”, so my first question was: “what is a digital twin?”"
}
,{
"title" : "When failure to share data costs lives",
"url" : "/academic/2021/05/a-data-sharing/",
"tag" : "academic, covid",
"category" : "academic",
"date" : "2021-05-16",
"excerpt" : "(June 2023: This draft article was inadvertently published some time in the past. It was written some time in 2020, I think, and I never finished it, and frankly I don’t know how it ever got published. I have just realised that it is “out there”, so rather than completely remove it, I have edited it slightly so the unfinished sentences are removed, and added a little coda on where we are now. However, it’s not my finest work.)"
}
,{
"title" : "The future of the ESGF",
"url" : "/computing/bigdata/2020/11/the-future-of-the-esgf/",
"tag" : "software, curation, esgf, cmip, cmip6",
"category" : "computingbigdata",
"date" : "2020-11-13",
"excerpt" : "I think most of the community who use climate projections and the products of climate model intercomparison projects depend in some way on the Earth System Grid Federation (ESGF). The ESGF consists of data nodes which are distributed globally, holding petabytes of data:"
}
,{
"title" : "Humans in climate models",
"url" : "/academic/2020/10/humans-in-climate-models/",
"tag" : "academic, climate, esm",
"category" : "academic",
"date" : "2020-10-26",
"excerpt" : "I recently had my attention drawn to Beckage et al, 2020 - “The Earth has humans, so why don’t our climate models?”"
}
,{
"title" : "ESIWACE2 Mid-Term Review",
"url" : "/talks/2020/10/Data-Systems/",
"tag" : "HPC, exascale, esiwace",
"category" : "talks",
"date" : "2020-10-21",
"excerpt" : "Data Systems at Scale"
}
,{
"title" : "On scientific software - reproducibility",
"url" : "/academic/2020/06/software4/",
"tag" : "software, curation, fair, publishing, journals",
"category" : "academic",
"date" : "2020-06-10",
"excerpt" : "Well I had to expose this series of blog posts before I had them all done because of this:"
}
,{
"title" : "On scientific software - definitions",
"url" : "/academic/2020/06/software3/",
"tag" : "software, curation",
"category" : "academic",
"date" : "2020-06-07",
"excerpt" : "So far in this series on scientific software, I discussed the notion that it might not be that easy to define quality for scientific code (and left some questions hanging about how we might do that), and introduced some scope questions>scope questions about what we mean by science code, and did that by producing a big diagram with lots of stuff."
}
,{
"title" : "On scientific software - scope",
"url" : "/academic/2020/06/software2/",
"tag" : "software, curation, FAIR, publishing, journals",
"category" : "academic",
"date" : "2020-06-01",
"excerpt" : "Last time I discussed the idea that the notion of “quality scientific software” wasn’t very helpful, insofar as scientific code quality is a bit like beauty, being “in the eye of the beholder” - but I concluded with a few questions that IMHO are more important than “is it quality?”, being:"
}
,{
"title" : "On scientific software - a beginning",
"url" : "/academic/2020/05/software1/",
"tag" : "software, curation, FAIR, publishing, journals",
"category" : "academic",
"date" : "2020-05-27",
"excerpt" : "All the opprobrium for Neil Ferguson’s epidemiological code has got to me. Enough that I’m going to commit myself to a series of blog posts, after a long silence here … (note that I have not committed to the interval period though!)"
}
,{
"title" : "Earthcube - What about Data?",
"url" : "/talks/2020/05/What-about-simulations/",
"tag" : "",
"category" : "talks",
"date" : "2020-05-05",
"excerpt" : "When (and how) should a simulation be Fair?Presentation: pdf (4 MB).After a reminder about FAIR (Findable, Accessible, Interoperable, Reusable - not Reproducible), I discuss some of the issues about applying these concepts to Simulation Data. Apart from the volume issues, we have to decide just what “Simulation Data” actually means, does it mean the entire workflow, or just the outputs. There are a huge variety of types of simulation, are they all important? Using the ES-DOC vocabularies, I point out that in practice simulations are not meaningfully reproducible - except in trivial cases, and actually it is more important to reproduce experiments - which is of course the heart of model intercomparison. At CEDA we have two decades of experience curating data, and we now use the JASMIN platform to provide a data commons to make data accessible. Lots of simulation data is analysed on JASMIN, alongside the CEDA archive, but for fifteen years we have made decisions about whether to archive simulation data based on (what was then) the BADC Numerical Model Data Policy. The key insight in developing that policy was that it’s relatively easy to decide what’s important, and what’s not important, but there is a lot of middle ground for which value judgements are important. We developed polices to decide on the important and not important data, and to guide the value judgements, and these are summarised here. Underlying all decisions are questions about affordabilty, and whether or not adequate metadata can be (or will be) produced. Without such metadata curation of simulation data becomes pointless. The bottom line is that not all simulation data should be FAIR, but that which needs to be FAIR needs to be well documented. Money matters as simulation data at scale is incredibly expensive."
}
,{
"title" : "The UM User's Workshop 2019 - Next Generation Modellin Systems",
"url" : "/talks/2019/06/umuw/",
"tag" : "HPC, exascale",
"category" : "talks",
"date" : "2019-06-18",
"excerpt" : "Challenges facing the modelling community"
}
,{
"title" : "ESIWACE2 Kick-Off",
"url" : "/talks/2019/03/Data-Systems/",
"tag" : "HPC, exascale, esiwace",
"category" : "talks",
"date" : "2019-03-12",
"excerpt" : "Data Systems at Scale"
}
,{
"title" : "NCAS Seminar",
"url" : "/talks/2019/03/end-of-climate-modelling/",
"tag" : "HPC, exascale",
"category" : "talks",
"date" : "2019-03-01",
"excerpt" : "The end of Climate Modelling as we know it"
}
,{
"title" : "Computer Science Seminar",
"url" : "/talks/2019/02/supercomputing-jasmin-diversity/",
"tag" : "HPC, exascale, JASMIN",
"category" : "talks",
"date" : "2019-02-11",
"excerpt" : "Supercomputers are no longer all the same and it will get worse; a climate modelling perspective"
}
,{
"title" : "NERC HPC Strategy",
"url" : "/talks/2018/11/NERC-HPC/",
"tag" : "HPC, exascale, JASMIN",
"category" : "talks",
"date" : "2018-11-27",
"excerpt" : "Trends and Context for Environmental HPC"
}
,{
"title" : "Extreme Data Workshop",
"url" : "/talks/2018/09/lawrence-extreme-data/",
"tag" : "data-science, exascale, JASMIN",
"category" : "talks",
"date" : "2018-09-18",
"excerpt" : "Beating Data Bottlenecks in Weather and Climate Science"
}
,{
"title" : "Open Meeting for Hydro-JULES - Next generation land-surface and hydrological predictions",
"url" : "/talks/2018/09/lawrence-hydrojules/",
"tag" : "jules, hydrology, climate, exascale, hydro-jules",
"category" : "talks",
"date" : "2018-09-10",
"excerpt" : "Building a Community Hydrological Model"
}
,{
"title" : "Workload Revisited",
"url" : "/management/2018/07/workload2/",
"tag" : "workload",
"category" : "management",
"date" : "2018-07-03",
"excerpt" : "In April I reported my first couple of months of workload anlaysis using the timing app. I’ve now done three more months, so was interested to look at how I’d done."
}
,{
"title" : "Getting to know an Ipad: Week One",
"url" : "/academic/2018/07/ipad/",
"tag" : "management",
"category" : "academic",
"date" : "2018-07-02",
"excerpt" : "I was an early adopter of Android tablets, and my most recent one (a 2014 Samsung Galaxy Note 10.1) was excellent. It’s still excellent, but runs an ancient version of Android – too ancient for my paranoia about information security, so it needed replacement."
}
,{
"title" : "JASMIN User Conference",
"url" : "/talks/2018/06/jasmin-uc/",
"tag" : "data-science, exascale, JASMIN",
"category" : "talks",
"date" : "2018-06-27",
"excerpt" : "The Changing Nature of JASMIN"
}
,{
"title" : "Data-Intensive weather and climate science",
"url" : "/talks/2018/06/climate-data-for-mpe/",
"tag" : "data-science, exascale",
"category" : "talks",
"date" : "2018-06-25",
"excerpt" : "Climate Data: Issues, Systems, and Opportunities"
}
,{
"title" : "EuroHPC Requirements Workshop",
"url" : "/talks/2018/06/EuroHPC-Requirements/",
"tag" : "data-science, esiwace, eurohpc, exascale",
"category" : "talks",
"date" : "2018-06-13",
"excerpt" : "EuroHPC: Requirements from Weather and Climate"
}
,{
"title" : "Workload - Where does Bryan spend his time?",
"url" : "/management/2018/04/workload/",
"tag" : "workload",
"category" : "management",
"date" : "2018-04-12",
"excerpt" : "I’ve always wondered where I spend my time. I keep records for some projects because one has to, but I’ve never found a way of recording interesting information easily enough. That changed recently: it turns out it is now possible to get a handle on where you spend time pretty easily, and at the same time learn about what applications you use most."
}
,{
"title" : "Data Sciences for Climate and Environment (Alan Turing Institute)",
"url" : "/talks/2018/03/data-science/",
"tag" : "data-science",
"category" : "talks",
"date" : "2018-03-26",
"excerpt" : "This talk covers a range of challenges for, and examples of, using (modern) data science techniques in Environmental Science (of course we have been using data science forever). Some issues with hardware and software environment are discussed."
}
,{
"title" : "Requirements for a global data infrastructure in support of CMIP6",
"url" : "/computing/2018/03/wip-paper/",
"tag" : "cmip6",
"category" : "computing",
"date" : "2018-03-22",
"excerpt" : "Our paper on the work of the WCRP Working Group on Climate Modeling (WGCM) Infrastructure Panel (WIP) wrecommendations for the global data infrastructure needed to support CMIP design, future growth and evolution has just appeared in GMD discussions.Please have a look and potentially contribute to the online review/discussion."
}
,{
"title" : "The Exascale Challenge to the UK - Part Tree - Capacity",
"url" : "/computing/2017/12/exascale3/",
"tag" : "exascale, cms, modelling_futures",
"category" : "computing",
"date" : "2017-12-31",
"excerpt" : "In this post we want to consider how much science we can do with the computers we currently have available (ARCHER and NEXCS), and in subsequent posts we’ll consider the route to exascale."
}
,{
"title" : "The CF Data Model Paper Appears",
"url" : "/academic/2017/12/cfdm-paper/",
"tag" : "cf, data_modelling, papers",
"category" : "academic",
"date" : "2017-12-20",
"excerpt" : "It’s always a nice present for any academic when a paper finally appears “in print”. After some months in public review, our paper introducing a formal data model for the Climate Forecast (CF) conventions (Hassell, 2017) has appeared in Geoscientific Model Development. Although CF was developed for content stored in NetCDF files, this data model has been developed in such a way that the logic can be used for CF content stored in other formats."
}
,{
"title" : "ESiWACE General Assembly",
"url" : "/talks/2017/12/esiwace-ga/",
"tag" : "esiwace, exascale",
"category" : "talks",
"date" : "2017-12-12",
"excerpt" : "The talk covers the “exploitability” component of the ESIWACE project. The work we describe is development of cost models for exascale HPC storage, plus new software to write and manage data at scale."
}
,{
"title" : "The Exascale Challenge to the UK - Part Two - Compute",
"url" : "/computing/2017/12/exascale2/",
"tag" : "exascale, cms, modelling_futures",
"category" : "computing",
"date" : "2017-12-10",
"excerpt" : "Last time I set up the generic exascale challenge, this time I want to discuss how it is playing out here and now for UK climate modelling. This long-ish post discusses computer hardare and benchmarking of our codes. The next one will discuss the implication for science."
}
,{
"title" : "The Exascale Challenge to the UK - Part One",
"url" : "/computing/2017/12/exascale/",
"tag" : "exascale, isenes, esiwace",
"category" : "computing",
"date" : "2017-12-05",
"excerpt" : "It is no secret that much of the progress in climate system modelling in recent decades has been on the back of the (then) inexorable increase in computational power associated with Moore’s Law - much of which came for free as chips got faster. However, now they’re not so much getting faster as we will get more processors and more processor types, and we have to be smarter about how we programme for them (Lawrence et al, . This is a problem for all of science!The US, Japan and China have been working on the problem for a while, and Europe is rapidly trying to catch up, but the “exascale problem” is all but invisible in the UK. While it’s obviously unfeasible for the UK to build an exascale computer on their own in the current economic and political climate, it’s absolutely the case that UK science and industry will have to work out how to use exascale computing, not least because the tools and techniques will percolate down eventually. However, even before the percolation, UK will have to exploit exascale computing to remain competitive, so what will that entail for us?"
}
,{
"title" : "Dr Matt Jones",
"url" : "/academic/2017/11/congrats2matt/",
"tag" : "teaching, hpc_io",
"category" : "academic",
"date" : "2017-11-28",
"excerpt" : "A good day for any academic - when one of her or his students passes their viva (oral examination) for their doctorate."
}
,{
"title" : "European Big Data Value Forum",
"url" : "/talks/2017/11/HPC_Bigdata/",
"tag" : "",
"category" : "talks",
"date" : "2017-11-22",
"excerpt" : "The Data Deluge in High Resolution Climate and Weather Simulation"
}
,{
"title" : "Science and the Digital Revolution - Data, Standards, and Integration",
"url" : "/computing/2017/11/science_and_the_digital_revolution/",
"tag" : "data, cmip, esgf",
"category" : "computing",
"date" : "2017-11-15",
"excerpt" : "I was asked to give a talk at this CODATA meeting which was aimed at developing a roadmap for: Mobilising community support and advice for discipline-based initiatives to develop online data capacities and services; Priorities for work on interdisciplinary data integration and flagship projects; Approaches to funding and coordination; and Issues of international data governance.Details of my talk are here."
}
,{
"title" : "Science and the Digital Revolution - Data, Standards, and Integration",
"url" : "/talks/2017/11/data_interop_lawrence/",
"tag" : "",
"category" : "talks",
"date" : "2017-11-14",
"excerpt" : "Data Interoperability and Integration: A climate modelling perspective"
}
,{
"title" : "Gung Ho Network Meeting",
"url" : "/talks/2017/07/ppp4_gungho/",
"tag" : "",
"category" : "talks",
"date" : "2017-07-18",
"excerpt" : "Performance, Portability, Productivity: Which two do you want?"
}
,{
"title" : "International Supercomputing (ISC) and JASMIN User Conferences",
"url" : "/talks/2017/06/lawrence_jasmin/",
"tag" : "",
"category" : "talks",
"date" : "2017-06-27",
"excerpt" : "I gave two versions of this talk, one at at the International Supercomputing Conference’s Workshop on HPC I/O in the data centre, and one at the 2017 JASMIN User’s Conference."
}
,{
"title" : "Week Eight",
"url" : "/diary/2017/02/week_eight/",
"tag" : "",
"category" : "diary",
"date" : "2017-02-27",
"excerpt" : "Spent nearly the entire week (and certainly more than a “European maximum 48-hours”) on things to do with ESIWACE (on the deliverable I mentioned last time, reporting for the EC, etc). I did have a fair chunk of one day on other things as also chaired a meeting of the advisory panel for our climate predictions for the copernicus climate data store project (CP4CDS, a project about deploying and maintaining sofware for a special ESGF data node to support climate services) - that took up the best part of day. However, that’s pretty much it for the week. Hard to believe we’re two months into the year - and I still haven’t come up for air and managed to create any significant blogging time."
}
,{
"title" : "Weeks Six and Seven",
"url" : "/diary/2017/02/weeks_six_and_seven/",
"tag" : "",
"category" : "diary",
"date" : "2017-02-19",
"excerpt" : "Two weeks to report this time, primarily because I had a day off on sick leave, and two days on real leave, so there’s only seven weekday workdays to talk about (althoughI have just spent bit chunks of both yesterday and today, that is, sat and sun, on work as well)."
}
,{
"title" : "Space and Open Plan Offices",
"url" : "/management/2017/02/space_and_open_plan_offices/",
"tag" : "management",
"category" : "management",
"date" : "2017-02-06",
"excerpt" : "I was paying a bit more attention to twitter this morning than usual (I’m hoping I’ll get some feedback on my analysis of citations that I posted yesterday). One thing that blew by was this headline the Washington Post:"Google got it wrong. The open-office trend is destroying the workplace.""
}
,{
"title" : "Week Five",
"url" : "/diary/2017/02/week_five/",
"tag" : "",
"category" : "diary",
"date" : "2017-02-05",
"excerpt" : "Nothing exciting to report. I spent nearly the entire week processing email, interacting with my team (both directly and on slack), and producing short things (e.g. a management level one pager on why NCAS should continue to support the CF conventions, and how). However, I did spend a wee bit of time trying to reinforce some of my observations around why storage is becoming a much bigger deal in environmental science. More on that next …"
}
,{
"title" : "Why so much data? Part I - The rise of direct numerical simulation",
"url" : "/computing/2017/02/so_much_data_one/",
"tag" : "",
"category" : "computing",
"date" : "2017-02-05",
"excerpt" : "Over the last decade or so, many in the scientific community (especially in the environmental sciences) have been surprised by the increased cost of data handling - not only in absolute terms, but in terms of the percentage of the cost of doing “normal science”.This increase has been problematic on a number of fronts: not only do individuals not always plan appropriately for managing their data storage and handling, but even where they have, institutions and funding agencies have themselves been surprised and not always too keen to pay up. After all, in a world where there is very little new money in real terms (or even none), an increase in one part of the budget needs to be offset elsewhere. So, the real consequence of these increases in storage cost have to be decreases in the amount of science done (fewer staff/instruments/computers), and that’s unpalatable even if it’s unavoidable. At the moment many choose to think it is avoidable, which is quite a feasible position if you’re an ostrich, but not so good if you’re responsible for delivering science!"
}
,{
"title" : "Week Four",
"url" : "/diary/2017/01/week_four/",
"tag" : "",
"category" : "diary",
"date" : "2017-01-29",
"excerpt" : "Not much to report for week four, since the work I did this week was mostly “processing email” - although we did submit the outline EPSRC bid, so one thing actually done (for now). The lack of anything else substantial was down to still being in the States in the early part of the week, coupled with a virulent bout of (probably) food poisoning which knocked me out for a couple of days."
}
,{
"title" : "Week Three",
"url" : "/diary/2017/01/week_three/",
"tag" : "meeting, isenes",
"category" : "diary",
"date" : "2017-01-23",
"excerpt" : "Quite a different week, for different reasons.Firstly, the work. Up before the larks on Monday to take my daughter to school and then on by Eurostar to Paris. Monday through Wednesday this week was the final general assembly for the IS-ENES2 project: the second “Infrastructure (to support) the European Network for Earth System Modelling”."
}
,{
"title" : "Week Two",
"url" : "/diary/2017/01/week_two/",
"tag" : "",
"category" : "diary",
"date" : "2017-01-15",
"excerpt" : "Another week, another load of paperwork written … another week that didn’t feel much like science except for the Intel bit which at least was interesting …"
}
,{
"title" : "Week One",
"url" : "/diary/2017/01/week_one/",
"tag" : "",
"category" : "diary",
"date" : "2017-01-08",
"excerpt" : "So this is the end of week one of 2017. What did I spend the week doing?"
}
,{
"title" : "Back to the Future - I think therefore I WILL blog",
"url" : "/diary/2017/01/back_to_the_future_-_i_think_therefore_i_will_blog/",
"tag" : "",
"category" : "diary",
"date" : "2017-01-07",
"excerpt" : "(with apologies to Descartes :-)"
}
,{
"title" : "NERC Town Hall Meeting on Data Centre Futures",
"url" : "/talks/2016/10/lawrence_infrastructure/",
"tag" : "",
"category" : "talks",
"date" : "2016-10-13",
"excerpt" : "Data Centre Technology to Support Environmental Science"
}
,{
"title" : "Meteorology meets Computer Science Symposium",
"url" : "/talks/2016/09/lawrence/",
"tag" : "",
"category" : "talks",
"date" : "2016-09-15",
"excerpt" : "Computer Science Issues in Environmental Infrastructure"
}
,{
"title" : "JASMIN User Conference",
"url" : "/talks/2016/06/jasmin_lawrence/",
"tag" : "",
"category" : "talks",
"date" : "2016-06-27",
"excerpt" : "Science Drivers: Why JASMIN?"
}
,{
"title" : "CEDA Vocbulary Meeting",
"url" : "/talks/2016/03/esdoc10minutes/",
"tag" : "",
"category" : "talks",
"date" : "2016-03-01",
"excerpt" : "A ten minute introduction to ES-DOC technology"
}
,{
"title" : "IS-ENES2 2nd General Assembly",
"url" : "/talks/2016/02/esdoc4cmip6/",
"tag" : "",
"category" : "talks",
"date" : "2016-02-18",
"excerpt" : "ESDOC for CMIP6"
}
,{
"title" : "International Computing in Atmospheric Science (ICAS)",
"url" : "/talks/2015/09/lawrence_icas15/",
"tag" : "",
"category" : "talks",
"date" : "2015-09-14",
"excerpt" : "UK academic infrastructure to support (big) environmental sciencePresentation: pdf (18 MB).Abstract: Modern environmental science requires the fusion of ever growing volumes of data from multiple simulation and observational platforms. In the UK we are investing in the infrastructure necessary to provide the generation, management, and analysis of the relevant datasets. This talk discusses the existing and planned hardware and software infrastructure required to support the (primarily) UK academic community in this endeavour, and relates it to key international endeavours at the European and global scale – including earth observation programmes such as the Copernicus Sentinel missions, the European Network for Earth Simulation, and the Earth System Grid Federation."
}
,{
"title" : "RCUK Cloud Workshop",
"url" : "/talks/2015/06/Lawrence_Cloud/",
"tag" : "",
"category" : "talks",
"date" : "2015-06-04",
"excerpt" : "Why Cloud? Earth Systems SciencePresentation: pdf (6 MB).Alternative title: Data Driven Science: Bringing Computation to the Data. This talk covered the background trends and described the JASMIN approach."
}
,{
"title" : "playing with docker",
"url" : "/computing/2015/05/playing_with_docker/",
"tag" : "docker",
"category" : "computing",
"date" : "2015-05-10",
"excerpt" : "From time to time I get a very short opportunity to try and do some science, and I find the context switching harder and harder. To that end, I want to make more use of ipython-notebook."
}
,{
"title" : "EGU",
"url" : "/talks/2015/04/Lawrence_Tyranny/",
"tag" : "",
"category" : "talks",
"date" : "2015-04-17",
"excerpt" : "Beating the tyranny of scale with a private cloud configured for Big DataPresentation: pdf (5 MB).At the last minute I found that I wasn’t able to attend, but Phil Kershaw gave my talk. The abstract is available here (pdf)."
}
,{
"title" : "A citation and provenance system for climate modelling",
"url" : "/computing/2015/03/a_citation_and_provenance_system_for_climate_modelling/",
"tag" : "cmip",
"category" : "computing",
"date" : "2015-03-02",
"excerpt" : "What would a modelling citation and provenance system need to do?"
}
,{
"title" : "Pagico Experience at week one.",
"url" : "/management/2015/02/pagico_experience_at_week_one/",
"tag" : "management",
"category" : "management",
"date" : "2015-02-01",
"excerpt" : "Ok, I promised to report my experience with pagico at week one."
}
,{
"title" : "Big Data and Extreme-Scale Computing (BDEC)",
"url" : "/talks/2015/01/lawrence_4slides/",
"tag" : "",
"category" : "talks",
"date" : "2015-01-30",
"excerpt" : "There were two back to back meetings organised as part of the 2015 Big Data and Extreme Computing meeting website. In the first, organised as part of the European Exascale Software Initiative (EESI), I gave a full talk, in the second, I provided a four page position paper with a four page exposition.It starts and Ends with Data: Towards exascale from an earth system science perspectivePresentation: pdf (7 MB).Six sections: the big picture, background trends, hardware issues, software issues, workflow, and a summary.Bringing Compute to the DataPresentation: pdf (3 MB).This was my main BDEC contribution. There was also a four pagesummary paper: pdf."
}
,{
"title" : "Pagico and getting things done",
"url" : "/management/2015/01/pagico_and_getting_things_done/",
"tag" : "management",
"category" : "management",
"date" : "2015-01-25",
"excerpt" : "Getting Things Done. GTD."
}
,{
"title" : "Leptoukh Lecture",
"url" : "/academic/2014/12/leptoukh_lecture/",
"tag" : "",
"category" : "academic",
"date" : "2014-12-18",
"excerpt" : "I was honoured by the informatics section of the American Geophysical Union this year by being awarded the Leptoukh Lecture. The abstract and talk itself are here."
}
,{
"title" : "AGU Fall Meeting",
"url" : "/talks/2014/12/lawrence_leptoukh/",
"tag" : "",
"category" : "talks",
"date" : "2014-12-17",
"excerpt" : "I was honoured to be the third recipient of the AGU Leptoukh Lecture awarded for significant contributions to informatics, computational, or data sciences."
}
,{
"title" : "Symposium on HPC and Data-Intensive Apps",
"url" : "/talks/2014/11/Lawrence_ICTP/",
"tag" : "",
"category" : "talks",
"date" : "2014-11-14",
"excerpt" : "Or to give it it’s full name: Symposium on HPC and Data-Intensive Applications in Earth Sciences: Challenges and Opportunities@ICTP, Trieste, Italy.I gave two talks at this meeting, the first in the HPC regular session, on behalf of my colleague Pier Luigi Vidale, on UPSCALE, the second a data keynote on day two.Weather and Climate modelling at the Petascale: achievements and perspectives. The roadmap to PRIMAVERAPresentation: pdf (37 MB).Abstract:Recent results and plans from the Joint Met Office/NERC High Resolution Climate Modelling programme are presented, along with a summary of recent and planned model developments. We show the influence of high resolution on a number of important atmospheric phenomena, highlighting both the roles of multiple groups in the work and the need for further resolution and complexity improvements in multiple models. We introduce plans for a project to do just that. A final point is that this work is highly demanding of both the supercomputing and subsequent analysis environments.Infrastructure for Environmental Supercomputing: beyond the HPC!Abstract:We begin by motivating the problems facing us in environmental simulations across scales: complex community interactions, and complex infrastructure. Looking forward we see the drive to increased resolution and complexity leading not only to compute issues, but even more severe data storage and handling issues. We worry about the software consequences before moving to the only possible solution, more and better collaboration, with shared infrastructure. To make progress requires moving past consideration of software interfaces alone to consider also the “collaboration” interfaces. We spend considerable time describing the JASMIN HPC data collaboration environment in the UK, before reaching the final conclusion: Getting our models to run on (new) supercomputers is hard. Getting them to run perfomantly is hard. Analysing, exploiting and archiving the data is (probably) now even harder!Presentation: pdf (22 MB )"
}
,{
"title" : "NERC ICT Current Awareness",
"url" : "/talks/2014/10/Lawrence_JASMIN/",
"tag" : "jasmin",
"category" : "talks",
"date" : "2014-10-07",
"excerpt" : "JASMIN - A NERC Data Analysis Environment"
}
,{
"title" : "Building your own JASMIN Virtual Machine",
"url" : "/computing/2014/08/building_your_own_jasmin_virtual_machine/",
"tag" : "jasmin",
"category" : "computing",
"date" : "2014-08-04",
"excerpt" : "I make a good deal of use of the JASMIN science virtual machines, but sometimes I want to just do something locally for testing. Fortunately you can build your own virtual machine using the “ JASMIN Analysis Platform” (JAP) to get the same base files."
}
,{
"title" : "simulation documents",
"url" : "/computing/environment/2014/08/simulation_documents/",
"tag" : "esdoc, metafor, cmip6",
"category" : "computingenvironment",
"date" : "2014-08-01",
"excerpt" : "In my last post I was discussing the relationship between the various elements of documentation necessary to describe the simulation workflow."
}
,{
"title" : "Updated version of the model documentation plots",
"url" : "/computing/2014/07/updated_version_of_the_model_documentation_plots/",
"tag" : "esdoc",
"category" : "computing",
"date" : "2014-07-29",
"excerpt" : "A couple of weeks ago, I outlined three figures to help describe the model documentation workflow and asked for a bit of feedback."
}
,{
"title" : "NCAS Science Conference, Bristol",
"url" : "/2014/07/ncas_science_conference-2C_bristol/",
"tag" : "",
"category" : "",
"date" : "2014-07-18",
"excerpt" : "In the middle of two days in Bristol for the NCAS science conference. Good to see what so many of my colleagues are up to (distributed as they are, across the UK)."
}
,{
"title" : "NCAS Science Meeting",
"url" : "/talks/2014/07/Lawrence_NCAS_Sci/",
"tag" : "",
"category" : "talks",
"date" : "2014-07-17",
"excerpt" : "The influence of Moore’s Law and friends on our computing environment!"
}
,{
"title" : "The vocabulary of documenting models",
"url" : "/computing/2014/07/the_vocabulary_of_documenting_models/",
"tag" : "esdoc, metafor",
"category" : "computing",
"date" : "2014-07-09",
"excerpt" : "Some time ago our European project Metafor migrated to a global project called es-doc. During the metafor project we put a lot of effort into trying to develop materials which describe what it is trying to achieve, but I think we never really got that right (despite a number of papers on the subject - e.g. Lawrence et al 2012 , Guilyardi et al 2013 etc)."
}
,{
"title" : "e-Research NZ",
"url" : "/talks/2014/07/e-researchNZ14/",
"tag" : "",
"category" : "talks",
"date" : "2014-07-02",
"excerpt" : "I gave three talks at this meeting:"
}
,{
"title" : "Accessing JASMIN from Android",
"url" : "/computing/2013/11/accessing_jasmin_from_android/",
"tag" : "jasmin",
"category" : "computing",
"date" : "2013-11-13",
"excerpt" : "I have a confession to make. I sometimes illicitly work using my android phone and legitimately with my android tablet. Sometimes that work involves wanting to ssh into real computers, and edit files on real computers. One real computer I want to access is JASMIN, and access to JASMIN is controlled by passphrase protected public/private key pairs."
}
,{
"title" : "vertical resolution",
"url" : "/environment/2013/09/vertical_resolution/",
"tag" : "dynamics, resolution",
"category" : "environment",
"date" : "2013-09-18",
"excerpt" : " Last week I pointed out that I wasn’t at all sure the analysis by LFR89 really applied at modern horizontal grid resolutions, since the vertical scales implied for quasi-geostrophic motion didn’t make sense."
}
,{
"title" : "zotero, zandy, greader, evernote, and me",
"url" : "/academic/2013/09/zotero-2C_zandy-2C_greader-2C_evernote-2C_and_me/",
"tag" : "zotero, evernote",
"category" : "academic",
"date" : "2013-09-11",
"excerpt" : "Quite a while ago (i.e. years), I decided that managing my bibliographic information in a bibtex file wasn’t working any longer. Back then I had a look at Mendeley and Zotero. I can’t really remember why, but I chose Zotero (I think it was a combination of how it worked for me, I played with both, and I didn’t like having to use their PDF viewr. I also had some worries about Mendeley and the software and information IPR … when Elsevier bought out Mendeley I felt vindicated on the latter.)"
}
,{
"title" : "Vertical and Horizontal Resolution",
"url" : "/environment/2013/09/vertical_and_horizontal_resolution/",
"tag" : "dynamics, resolution",
"category" : "environment",
"date" : "2013-09-09",
"excerpt" : "I’ve been delving in the literature a bit this week … considering model resolution and various issues around it. This post is by way of notes from my reading."
}
,{
"title" : "Storing and manipulating environmental big data with JASMIN",
"url" : "/computing/2013/08/storing_and_manipulating_environmental_big_data_with_jasmin/",
"tag" : "jasmin",
"category" : "computing",
"date" : "2013-08-28",
"excerpt" : "We’re pleased that our paper on the first year of JASMIN has been accepted by the IEEE BigData 2013 conference."
}
,{
"title" : "Gavin's Proposal",
"url" : "/computing/2013/08/gavins_proposal/",
"tag" : "cmip",
"category" : "computing",
"date" : "2013-08-23",
"excerpt" : "Gavin made some proposals in a comment on my blog for how he thought an easy automatic DOI system could be set up for CMIP data in ESGF."