Search

Your search keyword '"Hu Qingbao"' showing total 30 results
Start Over Author "Hu Qingbao"
30 results on '"Hu Qingbao"'

1. Using Kerberos Tokens in Distributed Computing System at IHEP

Author

Jiang Xiaowei, Guo Chaoqi, Hu Qingbao, Du Ran, Shi Jingyan, and Sun Gongxing

Subjects
Physics, QC1-999
Abstract

The token-based certification method is spreading in the distributed computing system of high energy physics. More and more software and middleware are supporting tokens as one of the certification methods. As an example, WLCG has upgraded all the services to support WLCG tokens [1]. In IHEP (Institute of High Energy Physics in China), the Kerberos [2] token has been used as the main certification method in the local cluster. Naturally, it is selected as the certification method in the distributed computing system. In this case, a set of toolkits were developed or introduced to use Kerberos tokens in the distributed computing system, including token producer, token repository, token transfer and token client engine. The token producer is responsible for creating a token and publishing the token file to the token repository. The token repository stores all the latest token files and a refresh service periodically renews the lifetime of those tokens stored in the token repository. The token transfer brings the token file to the worker node. The token client engine initializes the token environment and renews the token’s lifetime on the worker node. With these toolkits, the jobs can run in any worker node in any site and use the Kerberos token to access other services, such as EOS [3] and the XRootd [4] proxy service. In IHEP, the Kerberos toolkit has been deployed in the distributed computing system. Currently, three experiments (LHAASO [5], BES [6] and HERD [7]) are using Kerberos tokens to remotely access the data in EOS or Lustre [8].

Published
2024
Full Text
View/download PDF

2. Design and implementation of experimental data access security policy for HEPS container computing platform

Author

Hu Qingbao, Xu Jiping, Cheng Yaosong, Luo Qi, and Fu Shiyuan

Subjects
Physics, QC1-999
Abstract

China’s High-Energy Photon Source (HEPS), the first national highenergy synchrotron radiation light source, is under design and construction. In the future, at the first stage of HEPS, it is predicted that 24PB raw experimental data will be produced per month from 14 beamlines. Faced with such a huge scale of scientific data and diverse data analysis environments in light source disciplines, the HEPS scientific computing platform was designed and implemented based on container mirroring and dynamic orchestration technology to provide HEPS users with a data analysis environment. In this article, a data security access strategy is designed and evaluated for a scientific computing platform to ensure the security and efficiency of data access for users in the entire process of data analysis. First, the general situation of HEPS is introduced. Second, the challenges faced by the HEPS scientific computing system. Third, the architecture and service process of the scientific computing platform are described from the perspective of IT, some key technical implementations will be introduced in detail. Finally, the application effect of data access security policies on computing platforms will be demonstrated.

Published
2024
Full Text
View/download PDF

3. Evolution of the LHAASO Distributed Computing System based Cloud

Author

Huang Qiulan, Li Haibo, Cheng Yaodong, Shi Jingyan, Zheng Wei, and Hu Qingbao

Subjects
Physics, QC1-999
Abstract

In this paper we will describe the LHAASO distributed computing system based on virtualization and cloud computing technologies. Particularly, we discuss the key points of integrating distributed resources. A solution of integrating cross-domain resources is proposed, which adopt the Openstack+HTCondor to make the distributed resources work as a whole resource pool. A flexible resource scheduling strategy and a job scheduling policy are presented to realize the resource expansion on demand and the efficient job scheduling to remote sites transparently, so as to improve the overall resource utilization. We will also introduce the deployment of the computing system located in Daocheng, the LHAASO observation base using cloud-based architecture, which greatly helps to reduce the operation and maintenance cost as well as to make sure the system availability and stability. Finally, we will show running status of the system.

Published
2020
Full Text
View/download PDF

4. Machine Learning-based Anomaly Detection of Ganglia Monitoring Data in HEP Data Center

Author

Chen Juan, Wang Lu, and Hu Qingbao

Subjects
Physics, QC1-999
Abstract

This paper introduces a generic and scalable anomaly detection framework. Anomaly detection can improve operation and maintenance efficiency and assure experiments can be carried out effectively. The framework facilitates common tasks such as data sample building, retagging and visualization, deviation measurement and performance measurement for machine learning-based anomaly detection methods. The samples we used are sourced from Ganglia monitoring data. There are several anomaly detection methods to handle spatial and temporal anomalies within the framework. Finally, we show the rudimental application of the framework on Lustre distributed file systems in daily operation and maintenance.

Published
2020
Full Text
View/download PDF

5. A Lightweight Submission Frontend Toolkit HepJob

Author

JIANG Xiaowei, Du Ran, Shi Jingyan, Zou Jiaheng, and Hu Qingbao

Subjects
Physics, QC1-999
Abstract

A typical HEP Computing Center normally runs at least one batch system. As an example, at IHEP (Institute of High Energy Physics, Chinese Academy of Sciences), we’ve used three batch systems: PBS, HTCondor and SLURM. After running PBS as a local batch system for 10 years, we replaced it by HTCondor (for HTC) and SLURM (for HPC). During that period, problems came up on both user and admin sides. Introduction of the new batch systems implies necessity for users to acquire additional knowledge specific for every batch system, in particular, batch commands. In some cases, users have to use both HTCondor and SLURM in parallel. Furthermore, HTCondor and SLURM provide more functionality, which means more complicated usage mode, compared to the simple PBS commands. On admin side, HTCondor gives more freedom to users, which brings an additional challenge to site administrators. Site administrators have to find the solutions for many problems: preventing users from requesting the resources they are not allowed to use, checking if the required attributes are correct, deciding where requested resources are located (SLURM cluster, the cluster of the virtual machines, the remote sites, etc). To meet the above requirements, HepJob was designed and developed. HepJob provides a set of simple user commands, for example: hep_sub, hep_q, hep_rm, etc. In the submission process, HepJob checks all the attributes and ensures all attributes are correct; assigns proper resources to users (the user and group info is obtained from the management database); routes jobs to the target site; performs other steps as required. Users can start with HepJob very easily and administrators can take the necessary management actions in HepJob.

Published
2020
Full Text
View/download PDF

6. An automatic solution to make HTCondor more stable and easier

Author

Shi Jingyan, Zou Jiaheng, Hu Qingbao, Jiang Xiaowei, and Ou Ge

Subjects
Physics, QC1-999
Abstract

HTCondor has been widely adopted by HEP clusters to provide high-level scheduling performance. Unlike other schedulers, HTCondor provides loose management of the worker nodes. We developed a maintenance automation tool called “HTCondor MAT” that focuses on dynamic resource management and automatic error handling. A central database records all worker node information, which is sent to the worker node for the startd configuration. If an error happens for the worker node, the node information stored in the database is updated and the worker node is reconfigured with the new node information. The new configuration stops the startd from accepting error-related jobs until the worker node recovers. The MAT has been deployed in the IHEP HTC cluster to provide a central way to manage the worker nodes and remove the impacts of errors on the worker nodes automatically.

Published
2020
Full Text
View/download PDF

7. Research and Exploit of Resource Sharing Strategy at IHEP

Author

JIANG Xiaowei, Shi Jingyan, Zou Jiaheng, Hu Qingbao, Du Ran, and Sun Zhenyu

Subjects
Physics, QC1-999
Abstract

At IHEP (Institute of High Energy Physics, Chinese Academy of Sciences), computing resources are contributed by different experiments including BES, JUNO, DYW, HXMT, etc. The resources were divided into different partitions to satisfy the dedicated experiment data processing requirements. IHEP had a local Torqu&Maui cluster with 50 queues serving for above 10 experiments. The separated resource partitions leaded to imbalance resource load. In a typical situation, BES resource partition was quite busy without free slot but still with lots of jobs in idle, while JUNO resources are free and wasted seriously. After moving resources from Torque&Maui to HTCondor in 2016, job scheduling efficiency has been improved a lot. In order to balance the imbalance resource load, we designed an efficient sharing strategy to improve the overall resourceutilization. We created an unified pool shared by all experiments. For each experiment, resources are divided into two parts: dedicated resource and sharing resource. The slots in dedicated resource only run jobs from its own experiment, and the slots in sharing resource are shared by jobs from all experiments. Default ratio of dedicated resource to sharing resource is 1:4. To maximize the sharing effectiveness, the ratio is dynamically adjusted between 0:5 and 4:1 based on the number of jobs submitted by each experiment. We have developed a central control system to decide how many resources can be allocated to each experiment group. This system is implemented at two sides: server side and client side. A management database is built at server side, which is storing resource, group and experiment information. Once the sharing ratio needs to be adjusted, resource group will be changed and updated into database. The resource group information is published to the server buffer in real-time. The client periodically pulls resource group information from server buffer via https protocol And resource scheduling configuration at client side is changed based on the resource group information. With this method, share ratio can be modified and deployed dynamically. Sharing strategy is implemented with HTCondor. ClassAd mechanism and accounting-group in HTCondor facilitate to utilizethe sharing strategy at IHEP computing cluster. With the sharing strategy, resource usage has been improved dramatically.

Published
2019
Full Text
View/download PDF

8. Cyber security detection and monitoring at IHEP private cloud for web services

Author

Yan Tian, Zeng Shan, Qi Mengyao, Hu Qingbao, and Qi Fazhi

Subjects
Physics, QC1-999
Abstract

To improve hardware utilization and save manpower in system maintenance, most of the web services in IHEP have been migrated to a private cloud build upon OpenStack. However, cyber security attacks becomes a serious threats to the cloud progressively. Therefore, a cyber security detection and monitoring system is deployed for this cloud platform. This system collects various security related logs as data sources, and processes them in a framework composed of open source data store, analysis and visualization tools. With this system, security incidents and events can be handled in time and rapid response can be taken to protect cloud platform against cyber security threats.

Published
2019
Full Text
View/download PDF

9. Using Hadoop for High Energy Physics Data Analysis

Author

Huang, Qiulan, Wei, Zhanchen, Sun, Gongxing, Cheng, Yaodong, Cheng, Zhenjing, Hu, Qingbao, Goos, Gerhard, Founding Editor, Hartmanis, Juris, Founding Editor, Bertino, Elisa, Editorial Board Member, Gao, Wen, Editorial Board Member, Steffen, Bernhard, Editorial Board Member, Woeginger, Gerhard, Editorial Board Member, Yung, Moti, Editorial Board Member, Li, Jianhui, editor, Meng, Xiaofeng, editor, Zhang, Ying, editor, Cui, Wenjuan, editor, and Du, Zhihui, editor

Published
2019
Full Text
View/download PDF

10. Automated and Intelligent Data Migration Strategy in High Energy Physical Storage Systems

Author

Cheng, Zhenjing, Cheng, Yaodong, Wang, Lu, Hu, Qingbao, Li, Haibo, Xu, Qi, Chen, Gang, Goos, Gerhard, Founding Editor, Hartmanis, Juris, Founding Editor, Bertino, Elisa, Editorial Board Member, Gao, Wen, Editorial Board Member, Steffen, Bernhard, Editorial Board Member, Woeginger, Gerhard, Editorial Board Member, Yung, Moti, Editorial Board Member, Li, Jianhui, editor, Meng, Xiaofeng, editor, Zhang, Ying, editor, Cui, Wenjuan, editor, and Du, Zhihui, editor

Published
2019
Full Text
View/download PDF

15. The enhanced x-ray timing and polarimetry mission: eXTP: an update on its scientific cases, mission profile and development status

Author

Zhang, Shuang Nan, Santangelo, Andrea, Xu, Yupeng, Feroci, Marco, Hernanz, Margarita, Lu, Fangjun, Chen, Yong, Feng, Hua, Nandra, Kirpal, Jiang, Weichun, Svoboda, Jiri, Brandt, Søren, Schanne, Stéphane, Zand, Jean, Michalska, Malgorzata, Bozzo, Enrico, Kalemci, Emrah, Agudo, Ivan, Ahangarianabhari, Mahdi, Aitink-Kroes, Gabby, Ambrosi, Giovanni, Ambrosino, Filippo, An, Zhenghua, Perez Torres, Miguel Angel, Antonelli, Matias, Argan, Andrea, Babinec, Viktor, Baldini, Luca, Barbera, Marco, van Baren, Coen, Baudin, David, Bayer, Jörg, Bellazzini, Ronaldo, Bellutti, Pierluigi, Bertucci, Bruna, Bertuccio, Giuseppe, Bi, Xingzi, Boezio, Mirko, Bonvicini, Valter, Bonvicini, Walter, Bordas, Pol, Borghese, Alice, Borghi, Giacomo, Bouyjou, Florent, Bozkurt, Ayhan, Brez, Alessandro, Brienza, Daniele, Cadoux, Franck, Campana, Riccardo, Cao, Jiewei, Cao, Xuelei, Casares, Jorge, Cavazzuti, Elisabetta, Ceraudo, Francesco, Chen, Tianxiang, Chen, Wen, Chen, Can, Chen, Yupeng, Chen, Xin, Chen, Yehai, Chenevez, Jerome, Cheng, Yaodong, Cirrincione, Daniela, Civitani, Marta, Cong, Min, Zelati, Francesco Coti, Cui, Weiwei, Cui, Tao, Cui, Wei, Dai, Boyu, Dauser, Thomas, De Angelis, Nicolas, De Marco, Barbara, De Rosa, Alessandra, Monte, Ettore Del, Cosimo, Sergio Di, Diebold, Sebastian, Dilillo, Giuseppe, Ding, Fei, Dohnal, Roman, Dong, Zefang, Donnarumma, Immacolata, Dovciak, Michal, Du, Yuanyuan, Ducci, Lorenzo, Evangelista, Yuri, Fan, Qingmei, Favre, Yannick, Ferrés, Patrícia, Fiandrini, Emanuele, Ficorella, Francesco, Fuschino, Fabio, Gálvez, José Luis, Gao, Na, Gao, Min, Ge, Yuqiang, Ge, Mingyu, Gevin, Olivier, Grassi, Marco, Gu, Yudong, Gu, Quanying, Guan, Ju, Guedel, Manuel, Han, Xingbo, Han, Dawei, He, Huilin, He, Junwang, Hedderman, Paul, den Herder, Jan Willem, Hong, Bin, Hormaetxe, Ander, Hou, Dongjie, Hu, Zexun, Hu, Hao, Hu, Qingbao, Hu, Yu, Huang, Yue, Huang, Jiangjiang, Huang, Qiushi, Huo, Jia, Hynek, Richard, Iwasawa, Kazumi, Izzo, Lucca, Ji, Long, Jia, Shumei, Jiang, Bowen, Jiang, Wei, Jiang, Jiechen, Jiang, Xiaowei, Jiao, Yang, Jin, Ge, Jin, Fan, Jose, Jordi, Karas, Vladimir, Kennedy, Thomas, Kirsch, Christian, Kole, Merlin, Komarek, Martin, Kreykenbohm, Ingo, Kuiper, Lucien, Kuvvetli, Irfan, Labanti, Claudio, Latronico, Luca, Laubert, Phillip, Li, Tao, Li, Longhui, Li, Hong, Li, Duo, Li, Wei, Li, Maoshun, Li, Gang, Li, Xiaobo, Li, Chengkui, Li, Bing, Li, Haibo, Liang, Hong, Liao, Qiuyan, Liao, Jinyuan, Limousin, Olivier, Lin, Yanjian, Linares, Manuel, Liu, Rui, Liu, Yichen, Liu, Zhihao, Liu, Hongwei, Liu, Xiaojing, Liu, Xiaohua, Cicero, Ugo Lo, Loehring, Jens, Lombardi, Giovanni, Lorenz, Maximilian, Lu, Bing, Lucchesi, Leonardo, Luo, Tao, Luo, Laidan, Ma, Jia, Ma, Xiang, Macera, Daniele, Malcovati, Piero, Manfreda, Alberto, Paredes, Josep Maria, Marinucci, Andrea, Martindale, Adrian, Meidinger, Norbert, Mele, Filippo, Mendes, Vasco, Meng, Bin, Merkl, Martin, Meuris, Aline, Miniutti, Giovanni, Minuti, Massimo, Morbidini, Alfredo, Morgante, Gianluca, Muleri, Fabio, Munini, Riccardo, Mussolin, Lorenzo, Negri, Barbara, Nie, Jianyin, Novák, Petr, Nowosielski, Witold, Nuti, Alessio, Onat, Ahmet, Orleanski, Piotr, Orsini, Leonardo, Ottensamer, Roland, Ou, Ge, Pacciani, Luigi, Paltani, Stephane, Pan, Teng, Pan, Kai, Pareschi, Giovanni, Patruno, Alessandro, Pepponi, Giancarlo, Perinati, Emanuele, Piazzolla, Raffaele, Picciotto, Antonino, Piemonte, Claudio, Pinchera, Michele, Pliego, Samuel, Poutanen, Juri, Qi, Liqiang, Qi, Fazhi, Qiang, Pengfei, Qiao, Fangjian, Qiao, Zheng, Qiu, Xiangbiao, Qiu, Chengbo, Qu, Jinlu, Rachevski, Alexandre, Rashevskaia, Irina, Rea, Nanda, Ribo, Marc, de la Rie, Rob, Rodriguez, Jerome, Gil, Pablo Rodríguez, Sala, Gloria, Schanz, Thomas, Serafinelli, Roberto, Sgro, Carmelo, Shen, Zhengxiang, Sheng, Lizhi, Shi, Jingyan, Sironi, Giorgia, Skup, Konrad, Song, Liming, Song, Zeyu, Song, Jiangbo, Spandre, Gloria, Spiga, Daniele, Su, Yongquan, Sun, Liang, Sungur, Müberra, Svéda, Libor, Tagliaferri, Gianpiero, Tan, Ying, Tao, Lian, Tcherniak, Denis, Tenzer, Chris, Todaro, Michela, Tolos, Laura, Torok, Gabriel, Torres, Diego F., Trois, Alessio, Turhan, Onur, Uttley, Phil, Vacchi, Andrea, Virgilli, Enrico, Vogel, Channah, Walton, Dave, Wang, Jian, Wang, Xianqi, Wang, Bo, Wang, Langping, Wang, Xiaofeng, Wang, Dianlong, Wang, Yusa, Wang, Juan, Wang, Hao, Wang, Ruijie, Wang, Ping, Wang, Lingjun, Wang, Wenshuai, Wang, Tian, Wang, Haoyu, Wang, Zhanshan, Watts, Anna L., Wen, Xiangyang, Wilms, Jörn, Winter, Berend, Wu, Xin, Wu, Chao, Wu, Zhenyu, Wu, Qiong, Wu, Kaiji, Xiong, Hao, Xiong, Shaolin, Xu, Wei, Xu, Zhao, Xu, Jingjing, Xu, Xiongwei, Xue, Jiadai, Yan, Yongqing, Yan, Xingtao, Yang, Xianghui, Yang, Yanji, Yang, Xiongtao, Yang, Sheng, Yang, Jiawei, Yang, Yingquan, Yi, Shuxu, Yu, Yonggao, Yu, Ke, Zampa, Ganluigi, Zampa, Nicola, Zane, Silvia, Zdziarski, Andrzej A., Zhang, Long, Zhang, Shu, Zhang, Wenda, Zhang, Xiaoli, Zhang, Zhen, Zhang, Jiawei, Zhang, Fan, Zhang, Ziliang, Zhang, Dali, Zhang, Honglin, Zhang, Aimei, Zhang, Tong, Zhang, Hongmei, Zhang, Juan, Zhang, Liang, Zhang, Yueting, Zhao, Li, Zhao, Baosheng, Zhao, Zijian, Zhao, Xiaofan, Zhao, Haisheng, Zhao, Xiaoyun, Zheng, Shijie, Zhou, Yupeng, Zhou, Xiaohong, Zhu, Jiahuan, Zhu, Yuxuan, Zhu, Cheng, Zhu, Zhencai, Zorzi, Nicola, Zwart, Frans, den Herder, Jan-Willem A., Nikzad, Shouleh, Nakazawa, Kazuhiro, Chinese Academy of Sciences, Agenzia Spaziale Italiana, Ministerio de Ciencia e Innovación (España), Federal Ministry of Economics and Technology (Germany), and Foundation for Polish Science

Subjects
Equation of state, Extreme gravity, Black hole, X-ray timing, X-ray polarimetry, Neutron star, Extreme magnetism, Extreme density
Abstract

Space Telescopes and Instrumentation 2022: Ultraviolet to Gamma Ray (2022), Montreal, Jul 17-22, 2022.--Proceedings of SPIE - The International Society for Optical Engineering vol. 12181 Article number 121811W.-- Full list of authors: Zhang, Shuang-Nan; Santangelo, Andrea; Xu, Yupeng; Feroci, Marco; Hernanz, Margarita; Lu, Fangjun; Chen, Yong; Feng, Hua; Nandra, Kirpal; Jiang, Weichun; Svoboda, Jiri; Brandt, Soren; Schanne, Stephane; in't Zand, Jean; Michalska, Malgorzata; Bozzo, Enrico; Kalemci, Emrah; Agudo, Ivan; Ahangarianabhari, Mahdi; Aitink-Kroes, Gabby; Ambrosi, Giovanni; Ambrosino, Filippo; An, Zhenghua; Perez Torres, Miguel Angel; Antonelli, Matias; Argan, Andrea; Babinec, Viktor; Baldini, Luca; Barbera, Marco; van Baren, Coen; Baudin, David; Bayer, Joerg; Bellazzini, Ronaldo; Bellutti, Pierluigi; Bertucci, Bruna; Bertuccio, Giuseppe; Bi, Xingzi; Boezio, Mirko; Bonvicini, Valter; Bonvicini, Walter; Bordas, Pol; Borghese, Alice; Borghi, Giacomo; Bouyjou, Florent; Bozkurt, Ayhan; Brez, Alessandro; Brienza, Daniele; Cadoux, Franck; Campana, Riccardo; Cao, Jiewei; Cao, Xuelei; Casares, Jorge; Cavazzuti, Elisabetta; Ceraudo, Francesco; Chen, Tianxiang; Chen, Wen; Chen, Can; Chen, Yupeng; Chen, Xin; Chen, Yehai; Chenevez, Jerome; Cheng, Yaodong; Cirrincione, Daniela; Civitani, Marta; Cong, Min; Coti Zelati, Francesco; Cui, Weiwei; Cui, Tao; Cui, Wei; Dai, Boyu; Dauser, Thomas; De Angelis, Nicolas; De Marco, Barbara; De Rosa, Alessandra; Del Monte, Ettore; Di Cosimo, Sergio; Diebold, Sebastian; Dilillo, Giuseppe; Ding, Fei; Dohnal, Roman; Dong, Zefang; Donnarumma, Immacolata; Dovciak, Michal; Du, Yuanyuan; Ducci, Lorenzo; Evangelista, Yuri; Fan, Qingmei; Favre, Yannick; Ferres, Patricia; Fiandrini, Emanuele; Ficorella, Francesco; Fuschino, Fabio; Luis Galvez, Jose; Gao, Na; Gao, Min; Ge, Yuqiang; Ge, Mingyu; Gevin, Olivier; Grassi, Marco; Gu, Yudong; Gu, Quanying; Guan, Ju; Guedel, Manuel; Han, Xingbo; Han, Dawei; He, Huilin; He, Junwang; Hedderman, Paul; den Herder, Jan-Willem; Hong, Bin; Hormaetxe, Ander; Hou, Dongjie; Hu, Zexun; Hu, Hao; Hu, Qingbao; Hu, Yu; Huang, Yue; Huang, Jiangjiang; Huang, Qiushi; Huo, Jia; Hynek, Richard; Iwasawa, Kazumi; Izzo, Lucca; Ji, Long; Jia, Shumei; Jiang, Bowen; Jiang, Wei; Jiang, Jiechen; Jiang, Xiaowei; Jiao, Yang; Jin, Ge; Jin, Fan; Jose, Jordi; Karas, Vladimir; Kennedy, Thomas; Kirsch, Christian; Kole, Merlin; Komarek, Martin; Kreykenbohm, Ingo; Kuiper, Lucien; Kuvvetli, Irfan; Labanti, Claudio; Latronico, Luca; Laubert, Phillip; Li, Tao; Li, Longhui; Li, Hong; Li, Duo; Li, Wei; Li, Maoshun; Li, Gang; Li, Xiaobo; Li, Chengkui; Li, Bing; Li, Haibo; Liang, Hong; Liao, Qiuyan; Liao, Jinyuan; Limousin, Olivier; Lin, Yanjian; Linares, Manuel; Liu, Rui; Liu, Yichen; Liu, Zhihao; Liu, Hongwei; Liu, Xiaojing; Liu, Xiaohua; Lo Cicero, Ugo; Loehring, Jens; Lombardi, Giovanni; Lorenz, Maximilian; Lu, Bing; Lucchesi, Leonardo; Luo, Tao; Luo, Laidan; Ma, Jia; Ma, Xiang; Macera, Daniele; Malcovati, Piero; Manfreda, Alberto; Maria Paredes, Josep; Marinucci, Andrea; Martindale, Adrian; Meidinger, Norbert; Mele, Filippo; Mendes, Vasco; Meng, Bin; Merkl, Martin; Meuris, Aline; Miniutti, Giovanni; Minuti, Massimo; Morbidini, Alfredo; Morgante, Gianluca; Muleri, Fabio; Munini, Riccardo; Mussolin, Lorenzo; Negri, Barbara; Nie, Jianyin; Novak, Petr; Nowosielski, Witold; Nuti, Alessio; Onat, Ahmet; Orleanski, Piotr; Orsini, Leonardo; Ottensamer, Roland; Ou, Ge; Pacciani, Luigi; Paltani, Stephane; Pan, Teng; Pan, Kai; Pareschi, Giovanni; Patruno, Alessandro; Pepponi, Giancarlo; Perinati, Emanuele; Piazzolla, Raffaele; Picciotto, Antonino; Piemonte, Claudio; Pinchera, Michele; Pliego, Samuel; Poutanen, Juri; Qi, Liqiang; Qi, Fazhi; Qiang, Pengfei; Qiao, Fangjian; Qiao, Zheng; Qiu, Xiangbiao; Qiu, Chengbo; Qu, Jinlu; Rachevski, Alexandre; Rashevskaia, Irina; Rea, Nanda; Ribo, Marc; de la Rie, Rob; Rodriguez, Jerome; Rodriguez Gil, Pablo; Sala, Gloria; Schanz, Thomas; Serafinelli, Roberto; Sgro, Carmelo; Shen, Zhengxiang; Sheng, Lizhi; Shi, Jingyan; Sironi, Giorgia; Skup, Konrad; Song, Liming; Song, Zeyu; Song, Jiangbo; Spandre, Gloria; Spiga, Daniele; Su, Yongquan; Sun, Liang; Sungur, Muberra; Sveda, Libor; Tagliaferri, Gianpiero; Tan, Ying; Tao, Lian; Tcherniak, Denis; Tenzer, Chris; Todaro, Michela; Tolos, Laura; Torok, Gabriel; Torres, Diego F.; Trois, Alessio; Turhan, Onur; Uttley, Phil; Vacchi, Andrea; Virgilli, Enrico; Vogel, Channah; Walton, Dave; Wang, Jian; Wang, Xianqi; Wang, Bo; Wang, Langping; Wang, Xiaofeng; Wang, Dianlong; Wang, Yusa; Wang, Juan; Wang, Hao; Wang, Ruijie; Wang, Ping; Wang, Lingjun; Wang, Wenshuai; Wang, Tian; Wang, Haoyu; Wang, Zhanshan; Watts, Anna L.; Wen, Xiangyang; Wilms, Joern; Winter, Berend; Wu, Xin; Wu, Chao; Wu, Zhenyu; Wu, Qiong; Wu, Kaiji; Xiong, Hao; Xiong, Shaolin; Xu, Wei; Xu, Zhao; Xu, Jingjing; Xu, Xiongwei; Xue, Jiadai; Yan, Yongqing; Yan, Xingtao; Yang, Xianghui; Yang, Yanji; Yang, Xiongtao; Yang, Sheng; Yang, Jiawei; Yang, Yingquan; Yi, Shuxu; Yu, Yonggao; Yu, Ke; Zampa, Ganluigi; Zampa, Nicola; Zane, Silvia; Zdziarski, Andrzej A.; Zhang, Long; Zhang, Shu; Zhang, Wenda; Zhang, Xiaoli; Zhang, Zhen; Zhang, Jiawei; Zhang, Fan; Zhang, Ziliang; Zhang, Dali; Zhang, Honglin; Zhang, Aimei; Zhang, Tong; Zhang, Hongmei; Zhang, Juan; Zhang, Liang; Zhang, Yueting; Zhao, Li; Zhao, Baosheng; Zhao, Zijian; Zhao, Xiaofan; Zhao, Haisheng; Zhao, Xiaoyun; Zheng, Shijie; Zhou, Yupeng; Zhou, Xiaohong; Zhu, Jiahuan; Zhu, Yuxuan; Zhu, Cheng; Zhu, Zhencai; Zorzi, Nicola; Zwart, Frans, The enhanced X-ray Timing and Polarimetry mission (eXTP) is a flagship observatory for X-ray timing, spectroscopy and polarimetry developed by an International Consortium. Thanks to its very large collecting area, good spectral resolution and unprecedented polarimetry capabilities, eXTP will explore the properties of matter and the propagation of light in the most extreme conditions found in the Universe. eXTP will, in addition, be a powerful X-ray observatory. The mission will continuously monitor the X-ray sky, and will enable multiwavelength and multi-messenger studies. The mission is currently in phase B, which will be completed in the middle of 2022. © 2022 SPIE. All rights reserved., The Chinese team acknowledges support by the Chinese Academy of Sciences through the Strategic Priority Research Program, Grant No. XDA15020100. The Italian authors acknowledge funding support by the Italian Space Agency (under agreement n. 2020-3-HH.0 and ASI-INAF n.2017-14-H.O) and INFN (project XRO -X-ray Observatories). The Spanish authors acknowledge funding support from the MICIN/AEI grant PID2019-108709GB-I00. The German team acknowledges support from the Bundesministerium fur Wirtschaft und Technologie through the Deutsches Zentrum fur Luft-und Raumfahrt e.V. (DLR) under the grant number FKZ 50 OO 1902. The Polish team acknowledges FNP grant POIR.04.04.00-00-5C65/17. The Czech team acknowledges support from the GACR project 21-06825X. The Turkish team acknowledges support from the Turkish Space Agency., With funding from the Spanish government through the Severo Ochoa Centre of Excellence accreditation SEV-2017-0709

Published
2022

20. The enhanced X-ray Timing and Polarimetry mission – eXTP:an update on its scientific cases, mission profile and development status

Author

den Herder, Jan-Willem A., Nikzad, Shouleh, Nakazawa, Kazuhiro, Zhang, Shuang Nan, Santangelo, Andrea, Xu, Yupeng, Feroci, Marco, Hernanz, Margarita, Lu, Fangjun, Chen, Yong, Feng, Hua, Nandra, Kirpal, Jiang, Weichun, Svoboda, Jiri, Brandt, Søren, Schanne, Stéphane, Zand, Jean, Michalska, Malgorzata, Bozzo, Enrico, Kalemci, Emrah, Agudo, Ivan, Ahangarianabhari, Mahdi, Aitink-Kroes, Gabby, Ambrosi, Giovanni, Ambrosino, Filippo, An, Zhenghua, Perez Torres, Miguel Angel, Antonelli, Matias, Argan, Andrea, Babinec, Viktor, Baldini, Luca, Barbera, Marco, van Baren, Coen, Baudin, David, Bayer, Jörg, Bellazzini, Ronaldo, Bellutti, Pierluigi, Bertucci, Bruna, Bertuccio, Giuseppe, Bi, Xingzi, Boezio, Mirko, Bonvicini, Valter, Bonvicini, Walter, Bordas, Pol, Borghese, Alice, Borghi, Giacomo, Bouyjou, Florent, Bozkurt, Ayhan, Brez, Alessandro, Brienza, Daniele, Cadoux, Franck, Campana, Riccardo, Cao, Jiewei, Cao, Xuelei, Casares, Jorge, Cavazzuti, Elisabetta, Ceraudo, Francesco, Chen, Tianxiang, Chen, Wen, Chen, Can, Chen, Yupeng, Chen, Xin, Chen, Yehai, Chenevez, Jerome, Cheng, Yaodong, Cirrincione, Daniela, Civitani, Marta, Cong, Min, Zelati, Francesco Coti, Cui, Weiwei, Cui, Tao, Cui, Wei, Dai, Boyu, Dauser, Thomas, De Angelis, Nicolas, De Marco, Barbara, De Rosa, Alessandra, Monte, Ettore Del, Cosimo, Sergio Di, Diebold, Sebastian, Dilillo, Giuseppe, Ding, Fei, Dohnal, Roman, Dong, Zefang, Donnarumma, Immacolata, Dovciak, Michal, Du, Yuanyuan, Ducci, Lorenzo, Evangelista, Yuri, Fan, Qingmei, Favre, Yannick, Ferrés, Patrícia, Fiandrini, Emanuele, Ficorella, Francesco, Fuschino, Fabio, Gálvez, José Luis, Gao, Na, Gao, Min, Ge, Yuqiang, Ge, Mingyu, Gevin, Olivier, Grassi, Marco, Gu, Yudong, Gu, Quanying, Guan, Ju, Guedel, Manuel, Han, Xingbo, Han, Dawei, He, Huilin, He, Junwang, Hedderman, Paul, den Herder, Jan Willem, Hong, Bin, Hormaetxe, Ander, Hou, Dongjie, Hu, Zexun, Hu, Hao, Hu, Qingbao, Hu, Yu, Huang, Yue, Huang, Jiangjiang, Huang, Qiushi, Huo, Jia, Hynek, Richard, Iwasawa, Kazumi, Izzo, Lucca, Ji, Long, Jia, Shumei, Jiang, Bowen, Jiang, Wei, Jiang, Jiechen, Jiang, Xiaowei, Jiao, Yang, Jin, Ge, Jin, Fan, Jose, Jordi, Karas, Vladimir, Kennedy, Thomas, Kirsch, Christian, Kole, Merlin, Komarek, Martin, Kreykenbohm, Ingo, Kuiper, Lucien, Kuvvetli, Irfan, Labanti, Claudio, Latronico, Luca, Laubert, Phillip, Li, Tao, Li, Longhui, Li, Hong, Li, Duo, Li, Wei, Li, Maoshun, Li, Gang, Li, Xiaobo, Li, Chengkui, Li, Bing, Li, Haibo, Liang, Hong, Liao, Qiuyan, Liao, Jinyuan, Limousin, Olivier, Lin, Yanjian, Linares, Manuel, Liu, Rui, Liu, Yichen, Liu, Zhihao, Liu, Hongwei, Liu, Xiaojing, Liu, Xiaohua, Cicero, Ugo Lo, Loehring, Jens, Lombardi, Giovanni, Lorenz, Maximilian, Lu, Bing, Lucchesi, Leonardo, Luo, Tao, Luo, Laidan, Ma, Jia, Ma, Xiang, Macera, Daniele, Malcovati, Piero, Manfreda, Alberto, Paredes, Josep Maria, Marinucci, Andrea, Martindale, Adrian, Meidinger, Norbert, Mele, Filippo, Mendes, Vasco, Meng, Bin, Merkl, Martin, Meuris, Aline, Miniutti, Giovanni, Minuti, Massimo, Morbidini, Alfredo, Morgante, Gianluca, Muleri, Fabio, Munini, Riccardo, Mussolin, Lorenzo, Negri, Barbara, Nie, Jianyin, Novák, Petr, Nowosielski, Witold, Nuti, Alessio, Onat, Ahmet, Orleanski, Piotr, Orsini, Leonardo, Ottensamer, Roland, Ou, Ge, Pacciani, Luigi, Paltani, Stephane, Pan, Teng, Pan, Kai, Pareschi, Giovanni, Patruno, Alessandro, Pepponi, Giancarlo, Perinati, Emanuele, Piazzolla, Raffaele, Picciotto, Antonino, Piemonte, Claudio, Pinchera, Michele, Pliego, Samuel, Poutanen, Juri, Qi, Liqiang, Qi, Fazhi, Qiang, Pengfei, Qiao, Fangjian, Qiao, Zheng, Qiu, Xiangbiao, Qiu, Chengbo, Qu, Jinlu, Rachevski, Alexandre, Rashevskaia, Irina, Rea, Nanda, Ribo, Marc, de la Rie, Rob, Rodriguez, Jerome, Gil, Pablo Rodríguez, Sala, Gloria, Schanz, Thomas, Serafinelli, Roberto, Sgro, Carmelo, Shen, Zhengxiang, Sheng, Lizhi, Shi, Jingyan, Sironi, Giorgia, Skup, Konrad, Song, Liming, Song, Zeyu, Song, Jiangbo, Spandre, Gloria, Spiga, Daniele, Su, Yongquan, Sun, Liang, Sungur, Müberra, Svéda, Libor, Tagliaferri, Gianpiero, Tan, Ying, Tao, Lian, Tcherniak, Denis, Tenzer, Chris, Todaro, Michela, Tolos, Laura, Torok, Gabriel, Torres, Diego F., Trois, Alessio, Turhan, Onur, Uttley, Phil, Vacchi, Andrea, Virgilli, Enrico, Vogel, Channah, Walton, Dave, Wang, Jian, Wang, Xianqi, Wang, Bo, Wang, Langping, Wang, Xiaofeng, Wang, Dianlong, Wang, Yusa, Wang, Juan, Wang, Hao, Wang, Ruijie, Wang, Ping, Wang, Lingjun, Wang, Wenshuai, Wang, Tian, Wang, Haoyu, Wang, Zhanshan, Watts, Anna L., Wen, Xiangyang, Wilms, Jörn, Winter, Berend, Wu, Xin, Wu, Chao, Wu, Zhenyu, Wu, Qiong, Wu, Kaiji, Xiong, Hao, Xiong, Shaolin, Xu, Wei, Xu, Zhao, Xu, Jingjing, Xu, Xiongwei, Xue, Jiadai, Yan, Yongqing, Yan, Xingtao, Yang, Xianghui, Yang, Yanji, Yang, Xiongtao, Yang, Sheng, Yang, Jiawei, Yang, Yingquan, Yi, Shuxu, Yu, Yonggao, Yu, Ke, Zampa, Ganluigi, Zampa, Nicola, Zane, Silvia, Zdziarski, Andrzej A., Zhang, Long, Zhang, Shu, Zhang, Wenda, Zhang, Xiaoli, Zhang, Zhen, Zhang, Jiawei, Zhang, Fan, Zhang, Ziliang, Zhang, Dali, Zhang, Honglin, Zhang, Aimei, Zhang, Tong, Zhang, Hongmei, Zhang, Juan, Zhang, Liang, Zhang, Yueting, Zhao, Li, Zhao, Baosheng, Zhao, Zijian, Zhao, Xiaofan, Zhao, Haisheng, Zhao, Xiaoyun, Zheng, Shijie, Zhou, Yupeng, Zhou, Xiaohong, Zhu, Jiahuan, Zhu, Yuxuan, Zhu, Cheng, Zhu, Zhencai, Zorzi, Nicola, Zwart, Frans, den Herder, Jan-Willem A., Nikzad, Shouleh, Nakazawa, Kazuhiro, Zhang, Shuang Nan, Santangelo, Andrea, Xu, Yupeng, Feroci, Marco, Hernanz, Margarita, Lu, Fangjun, Chen, Yong, Feng, Hua, Nandra, Kirpal, Jiang, Weichun, Svoboda, Jiri, Brandt, Søren, Schanne, Stéphane, Zand, Jean, Michalska, Malgorzata, Bozzo, Enrico, Kalemci, Emrah, Agudo, Ivan, Ahangarianabhari, Mahdi, Aitink-Kroes, Gabby, Ambrosi, Giovanni, Ambrosino, Filippo, An, Zhenghua, Perez Torres, Miguel Angel, Antonelli, Matias, Argan, Andrea, Babinec, Viktor, Baldini, Luca, Barbera, Marco, van Baren, Coen, Baudin, David, Bayer, Jörg, Bellazzini, Ronaldo, Bellutti, Pierluigi, Bertucci, Bruna, Bertuccio, Giuseppe, Bi, Xingzi, Boezio, Mirko, Bonvicini, Valter, Bonvicini, Walter, Bordas, Pol, Borghese, Alice, Borghi, Giacomo, Bouyjou, Florent, Bozkurt, Ayhan, Brez, Alessandro, Brienza, Daniele, Cadoux, Franck, Campana, Riccardo, Cao, Jiewei, Cao, Xuelei, Casares, Jorge, Cavazzuti, Elisabetta, Ceraudo, Francesco, Chen, Tianxiang, Chen, Wen, Chen, Can, Chen, Yupeng, Chen, Xin, Chen, Yehai, Chenevez, Jerome, Cheng, Yaodong, Cirrincione, Daniela, Civitani, Marta, Cong, Min, Zelati, Francesco Coti, Cui, Weiwei, Cui, Tao, Cui, Wei, Dai, Boyu, Dauser, Thomas, De Angelis, Nicolas, De Marco, Barbara, De Rosa, Alessandra, Monte, Ettore Del, Cosimo, Sergio Di, Diebold, Sebastian, Dilillo, Giuseppe, Ding, Fei, Dohnal, Roman, Dong, Zefang, Donnarumma, Immacolata, Dovciak, Michal, Du, Yuanyuan, Ducci, Lorenzo, Evangelista, Yuri, Fan, Qingmei, Favre, Yannick, Ferrés, Patrícia, Fiandrini, Emanuele, Ficorella, Francesco, Fuschino, Fabio, Gálvez, José Luis, Gao, Na, Gao, Min, Ge, Yuqiang, Ge, Mingyu, Gevin, Olivier, Grassi, Marco, Gu, Yudong, Gu, Quanying, Guan, Ju, Guedel, Manuel, Han, Xingbo, Han, Dawei, He, Huilin, He, Junwang, Hedderman, Paul, den Herder, Jan Willem, Hong, Bin, Hormaetxe, Ander, Hou, Dongjie, Hu, Zexun, Hu, Hao, Hu, Qingbao, Hu, Yu, Huang, Yue, Huang, Jiangjiang, Huang, Qiushi, Huo, Jia, Hynek, Richard, Iwasawa, Kazumi, Izzo, Lucca, Ji, Long, Jia, Shumei, Jiang, Bowen, Jiang, Wei, Jiang, Jiechen, Jiang, Xiaowei, Jiao, Yang, Jin, Ge, Jin, Fan, Jose, Jordi, Karas, Vladimir, Kennedy, Thomas, Kirsch, Christian, Kole, Merlin, Komarek, Martin, Kreykenbohm, Ingo, Kuiper, Lucien, Kuvvetli, Irfan, Labanti, Claudio, Latronico, Luca, Laubert, Phillip, Li, Tao, Li, Longhui, Li, Hong, Li, Duo, Li, Wei, Li, Maoshun, Li, Gang, Li, Xiaobo, Li, Chengkui, Li, Bing, Li, Haibo, Liang, Hong, Liao, Qiuyan, Liao, Jinyuan, Limousin, Olivier, Lin, Yanjian, Linares, Manuel, Liu, Rui, Liu, Yichen, Liu, Zhihao, Liu, Hongwei, Liu, Xiaojing, Liu, Xiaohua, Cicero, Ugo Lo, Loehring, Jens, Lombardi, Giovanni, Lorenz, Maximilian, Lu, Bing, Lucchesi, Leonardo, Luo, Tao, Luo, Laidan, Ma, Jia, Ma, Xiang, Macera, Daniele, Malcovati, Piero, Manfreda, Alberto, Paredes, Josep Maria, Marinucci, Andrea, Martindale, Adrian, Meidinger, Norbert, Mele, Filippo, Mendes, Vasco, Meng, Bin, Merkl, Martin, Meuris, Aline, Miniutti, Giovanni, Minuti, Massimo, Morbidini, Alfredo, Morgante, Gianluca, Muleri, Fabio, Munini, Riccardo, Mussolin, Lorenzo, Negri, Barbara, Nie, Jianyin, Novák, Petr, Nowosielski, Witold, Nuti, Alessio, Onat, Ahmet, Orleanski, Piotr, Orsini, Leonardo, Ottensamer, Roland, Ou, Ge, Pacciani, Luigi, Paltani, Stephane, Pan, Teng, Pan, Kai, Pareschi, Giovanni, Patruno, Alessandro, Pepponi, Giancarlo, Perinati, Emanuele, Piazzolla, Raffaele, Picciotto, Antonino, Piemonte, Claudio, Pinchera, Michele, Pliego, Samuel, Poutanen, Juri, Qi, Liqiang, Qi, Fazhi, Qiang, Pengfei, Qiao, Fangjian, Qiao, Zheng, Qiu, Xiangbiao, Qiu, Chengbo, Qu, Jinlu, Rachevski, Alexandre, Rashevskaia, Irina, Rea, Nanda, Ribo, Marc, de la Rie, Rob, Rodriguez, Jerome, Gil, Pablo Rodríguez, Sala, Gloria, Schanz, Thomas, Serafinelli, Roberto, Sgro, Carmelo, Shen, Zhengxiang, Sheng, Lizhi, Shi, Jingyan, Sironi, Giorgia, Skup, Konrad, Song, Liming, Song, Zeyu, Song, Jiangbo, Spandre, Gloria, Spiga, Daniele, Su, Yongquan, Sun, Liang, Sungur, Müberra, Svéda, Libor, Tagliaferri, Gianpiero, Tan, Ying, Tao, Lian, Tcherniak, Denis, Tenzer, Chris, Todaro, Michela, Tolos, Laura, Torok, Gabriel, Torres, Diego F., Trois, Alessio, Turhan, Onur, Uttley, Phil, Vacchi, Andrea, Virgilli, Enrico, Vogel, Channah, Walton, Dave, Wang, Jian, Wang, Xianqi, Wang, Bo, Wang, Langping, Wang, Xiaofeng, Wang, Dianlong, Wang, Yusa, Wang, Juan, Wang, Hao, Wang, Ruijie, Wang, Ping, Wang, Lingjun, Wang, Wenshuai, Wang, Tian, Wang, Haoyu, Wang, Zhanshan, Watts, Anna L., Wen, Xiangyang, Wilms, Jörn, Winter, Berend, Wu, Xin, Wu, Chao, Wu, Zhenyu, Wu, Qiong, Wu, Kaiji, Xiong, Hao, Xiong, Shaolin, Xu, Wei, Xu, Zhao, Xu, Jingjing, Xu, Xiongwei, Xue, Jiadai, Yan, Yongqing, Yan, Xingtao, Yang, Xianghui, Yang, Yanji, Yang, Xiongtao, Yang, Sheng, Yang, Jiawei, Yang, Yingquan, Yi, Shuxu, Yu, Yonggao, Yu, Ke, Zampa, Ganluigi, Zampa, Nicola, Zane, Silvia, Zdziarski, Andrzej A., Zhang, Long, Zhang, Shu, Zhang, Wenda, Zhang, Xiaoli, Zhang, Zhen, Zhang, Jiawei, Zhang, Fan, Zhang, Ziliang, Zhang, Dali, Zhang, Honglin, Zhang, Aimei, Zhang, Tong, Zhang, Hongmei, Zhang, Juan, Zhang, Liang, Zhang, Yueting, Zhao, Li, Zhao, Baosheng, Zhao, Zijian, Zhao, Xiaofan, Zhao, Haisheng, Zhao, Xiaoyun, Zheng, Shijie, Zhou, Yupeng, Zhou, Xiaohong, Zhu, Jiahuan, Zhu, Yuxuan, Zhu, Cheng, Zhu, Zhencai, Zorzi, Nicola, and Zwart, Frans

Abstract

The enhanced X-ray Timing and Polarimetry mission (eXTP) is a flagship observatory for X-ray timing, spectroscopy and polarimetry developed by an International Consortium. Thanks to its very large collecting area, good spectral resolution and unprecedented polarimetry capabilities, eXTP will explore the properties of matter and the propagation of light in the most extreme conditions found in the Universe. eXTP will, in addition, be a powerful X-ray observatory. The mission will continuously monitor the X-ray sky, and will enable multiwavelength and multi-messenger studies. The mission is currently in phase B, which will be completed in the middle of 2022.

Published
2022

22. Evolution of the LHAASO Distributed Computing System based Cloud

Author

Jingyan Shi, Hu Qingbao, Yaodong Cheng, Haibo Li, Qiulan Huang, and Wei Zheng

Subjects
Job scheduler, business.industry, Distributed computing, Physics, QC1-999, Stability (learning theory), Cloud computing, Virtualization, computer.software_genre, Resource (project management), Software deployment, Key (cryptography), Architecture, business, computer
Abstract

In this paper we will describe the LHAASO distributed computing system based on virtualization and cloud computing technologies. Particularly, we discuss the key points of integrating distributed resources. A solution of integrating cross-domain resources is proposed, which adopt the Openstack+HTCondor to make the distributed resources work as a whole resource pool. A flexible resource scheduling strategy and a job scheduling policy are presented to realize the resource expansion on demand and the efficient job scheduling to remote sites transparently, so as to improve the overall resource utilization. We will also introduce the deployment of the computing system located in Daocheng, the LHAASO observation base using cloud-based architecture, which greatly helps to reduce the operation and maintenance cost as well as to make sure the system availability and stability. Finally, we will show running status of the system.

Published
2020

23. Enhanced X-ray Timing and Polarimetry mission: eXTP: an update on its scientific cases, mission profile and development status

Author

den Herder, Jan-Willem A., Nikzad, Shouleh, Nakazawa, Kazuhiro, Zhang, Shuang-Nan, Santangelo, Andrea, Xu, Yupeng, Feroci, Marco, Hernanz, Margarita, Lu, Fangjun, Chen, Yong, Feng, Hua, Nandra, Kirpal, Jiang, Weichun, Svoboda, Jiri, Brandt, Søren, Schanne, Stéphane, in't Zand, Jean, Michalska, Malgosia, Bozzo, Enrico, Kalemci, Emrah, Agudo, Ivan, Ahangarianabhari, Mahdi, Aitink-Kroes, Gabby, An, Zhenghua, Cao, Jiewei, Cao, Xuelei, Chen, Tianxiang, Chen, Can, Chen, Yupeng, Cheng, Yaodong, Cong, Min, Cui, Weiwei, Cui, Tao, Wu, Zhenyu, Liu, Yichen, Su, Yongquan, Wang, Jian, Zhang, Zhen, Jin, Ge, Li, Longhui, Qiu, Xiangbiao, Lin, Yanjian, Li, Tao, Zhang, Jiawei, Wu, Chao, Xu, Wei, Hu, Zexun, Xu, Zhao, Qiao, Fangjian, Pan, Kai, Zhang, Shu, Song, Liming, He, Huilin, Zhang, Fan, Liu, Hongwei, Liu, Xiaojing, Yang, Yanji, Song, Zeyu, Zhang, Jiawei, Yu, Ke, Wang, Yusa, Li, Wei, Han, Dawei, Wang, Juan, Zhang, Ziliang, Wang, Hao, Zhang, Dali, Gao, Min, Ma, Jia, Huo, Jia, Li, Maoshun, Hou, Dongjie, Yang, Xiongtao, Zhao, Zijian, Zhao, Xiaofan, Xu, Jingjing, Luo, Laidan, Zhu, Yuxuan, Zhang, Honglin, Liu, Xiaohua, Gu, Yudong, Du, Yuanyuan, Yang, Sheng, Sun, Liang, Jiang, Jiechen, Yang, Jiawei, Dong, Zefang, Dai, Boyu, Jiao, Yang, Wen, Xiangyang, Meng, Bin, Zhang, Aimei, Wang, Ruijie, Zhang, Tong, Lu, Bing, Gao, Na, Xu, Xiongwei, Luo, Tao, Qi, Liqiang, Li, Gang, Qu, Jinlu, Xiong, Shaolin, Tao, Liao, Jia, Shumei, Ge, Mingyu, Zheng, Shijie, Li, Xiaobo, Ma, Xiang, Huang, Yue, Li, Chengkui, Nie, Jianyin, Zhao, Haisheng, Guan, Ju, Liao, Jinyuan, Zhang, Hongmei, Zhang, Juan, Wang, Ping, Zhao, Xiaoyun, Wang, Lingjun, Zhang, Liang, Yi, Shuxu, Li, Bing, Tan, Ying, Qi, Fazhi, Wang, Wenshuai, Ou, Ge, Hu, Hao, Shi, Jingyan, Hu, Qingbao, Jiang, Xiaowei, Li, Haibo, Hu, Yu, Jiang, Bowen, Tenzer, Chris, Perinati, Emanuele, and Pliego, Samuel

Published
2022
Full Text
View/download PDF

24. Automated and Intelligent Data Migration Strategy in High Energy Physical Storage Systems

Author

Zhenjing Cheng, Haibo Li, Qi Xu, Hu Qingbao, Lu Wang, Cheng Yaodong, and Gang Chen

Subjects
Data access, Software, business.industry, Process (engineering), Deep learning, Distributed computing, Data management, Computer data storage, Volume (computing), Artificial intelligence, business, Data migration
Abstract

As a data-intensive computing application, high-energy physics requires to process and store massive data at the PB or EB level. It requires high performance data access and large volume of data storage as well. Some enterprises and research organizations are beginning to use tiered storage architectures, using tapes, disks or solid drives at the same time to reduce hardware purchase costs and power consumption. Tiered storage requires data management software to migrate less active data to lower cost storage devices. Thus an automated data migration strategy is very necessary. Data access requests are driven by the behavior of users or programs. There must be associations between different files that are accessed consecutively. This paper proposes a method to predict the heat of data access and use data heat trend as the basis criteria for data migration. This paper proposes a deep learning algorithm model to predict the evolution trend of data access heat. This paper discussed the implementation of some initial parts of the system. Then some preliminary experiments are conducted with these parts.

Published
2019

25. Using Hadoop for High Energy Physics Data Analysis

Author

Zhenjing Cheng, Gongxing Sun, Qiulan Huang, Zhanchen Wei, Yaodong Cheng, and Hu Qingbao

Subjects
File system, Particle physics, business.industry, Computer science, Interface (computing), Parallel algorithm, computer.software_genre, Software, Scalability, Key (cryptography), business, Throughput (business), computer, Computer Science::Databases, Computer Science::Distributed, Parallel, and Cluster Computing, Random access
Abstract

With the development of the new generation of High Energy Physics (HEP) experiments, huge amounts of data are being generated. Efficient parallel algorithms/frameworks and High IO throughput are key to meet the scalability and performance requirements of HEP offline data analysis. Though Hadoop has gained a lot of attention from scientific community for its scalability and parallel computing framework for large data sets, it’s still difficult to make HEP data processing tasks run directly on Hadoop. In this paper we investigate the application of Hadoop to make HEP jobs run on it transparently. Particularly, we discuss a new mechanism to support HEP software to random access data in HDFS. Because HDFS is streaming data stored only supporting sequential write and append. It cannot satisfy HEP jobs to random access data. This new feature allows the Map/Reduce tasks to random read/write on the local file system on data nodes instead of using Hadoop data streaming interface. This makes HEP jobs run on Hadoop possible. We also develop diverse MapReduce model for HEP jobs such as Corsika simulation, ARGO detector simulation and Medea++ reconstruction. And we develop a toolkit for users to submit/query/remove jobs. In addition, we provide cluster monitoring and account system to benefit to the system availability. This work has been in production for HEP experiment to gain about 40,000 CPU hours per month since September, 2016.

Published
2019

26. An automatic solution to make HTCondor more stable and easier.

Author

Doglioni, C., Kim, D., Stewart, G.A., Silvestris, L., Jackson, P., Kamleh, W., Shi, Jingyan, Zou, Jiaheng, Hu, Qingbao, Jiang, Xiaowei, and Ou, Ge

Subjects
COMPUTER scheduling, RESOURCE management, INFORMATION storage & retrieval systems, SOFTWARE configuration management, WEB portals, LINUX operating systems
Abstract

HTCondor has been widely adopted by HEP clusters to provide high-level scheduling performance. Unlike other schedulers, HTCondor provides loose management of the worker nodes. We developed a maintenance automation tool called "HTCondor MAT" that focuses on dynamic resource management and automatic error handling. A central database records all worker node information, which is sent to the worker node for the startd configuration. If an error happens for the worker node, the node information stored in the database is updated and the worker node is reconfigured with the new node information. The new configuration stops the startd from accepting error-related jobs until the worker node recovers. The MAT has been deployed in the IHEP HTC cluster to provide a central way to manage the worker nodes and remove the impacts of errors on the worker nodes automatically. [ABSTRACT FROM AUTHOR]

Published
2020
Full Text
View/download PDF

27. Machine Learning-based Anomaly Detection of Ganglia Monitoring Data in HEP Data Center.

Author

Doglioni, C., Kim, D., Stewart, G.A., Silvestris, L., Jackson, P., Kamleh, W., Chen, Juan, Wang, Lu, and Hu, Qingbao

Subjects
ANOMALY detection (Computer security), MACHINE learning, DATA libraries, DATA visualization, ELECTRONIC file management
Abstract

This paper introduces a generic and scalable anomaly detection framework. Anomaly detection can improve operation and maintenance efficiency and assure experiments can be carried out effectively. The framework facilitates common tasks such as data sample building, retagging and visualization, deviation measurement and performance measurement for machine learning-based anomaly detection methods. The samples we used are sourced from Ganglia monitoring data. There are several anomaly detection methods to handle spatial and temporal anomalies within the framework. Finally, we show the rudimental application of the framework on Lustre distributed file systems in daily operation and maintenance. [ABSTRACT FROM AUTHOR]

Published
2020
Full Text
View/download PDF

28. Evolution of the LHAASO Distributed Computing System based Cloud.

Author

Doglioni, C., Kim, D., Stewart, G.A., Silvestris, L., Jackson, P., Kamleh, W., Huang, Qiulan, Li, Haibo, Cheng, Yaodong, Shi, Jingyan, Zheng, Wei, and Hu, Qingbao

Subjects
DISTRIBUTED computing, COMPUTER scheduling, DATA integration, COMPUTER architecture, CLOUD computing, MAINTENANCE costs
Abstract

In this paper we will describe the LHAASO distributed computing system based on virtualization and cloud computing technologies. Particularly, we discuss the key points of integrating distributed resources. A solution of integrating cross-domain resources is proposed, which adopt the Openstack+HTCondor to make the distributed resources work as a whole resource pool. A flexible resource scheduling strategy and a job scheduling policy are presented to realize the resource expansion on demand and the efficient job scheduling to remote sites transparently, so as to improve the overall resource utilization. We will also introduce the deployment of the computing system located in Daocheng, the LHAASO observation base using cloud-based architecture, which greatly helps to reduce the operation and maintenance cost as well as to make sure the system availability and stability. Finally, we will show running status of the system. [ABSTRACT FROM AUTHOR]

Published
2020
Full Text
View/download PDF

29. A Lightweight Submission Frontend Toolkit HepJob.

Author

Doglioni, C., Kim, D., Stewart, G.A., Silvestris, L., Jackson, P., Kamleh, W., JIANG, Xiaowei, Du, Ran, Shi, Jingyan, Zou, Jiaheng, and Hu, Qingbao

Subjects
PARTICLE physics, PYTHON programming language, DATA management, COMPUTER operating systems, VIRTUAL machine systems
Abstract

A typical HEP Computing Center normally runs at least one batch system. As an example, at IHEP (Institute of High Energy Physics, Chinese Academy of Sciences), we've used three batch systems: PBS, HTCondor and SLURM. After running PBS as a local batch system for 10 years, we replaced it by HTCondor (for HTC) and SLURM (for HPC). During that period, problems came up on both user and admin sides. Introduction of the new batch systems implies necessity for users to acquire additional knowledge specific for every batch system, in particular, batch commands. In some cases, users have to use both HTCondor and SLURM in parallel. Furthermore, HTCondor and SLURM provide more functionality, which means more complicated usage mode, compared to the simple PBS commands. On admin side, HTCondor gives more freedom to users, which brings an additional challenge to site administrators. Site administrators have to find the solutions for many problems: preventing users from requesting the resources they are not allowed to use, checking if the required attributes are correct, deciding where requested resources are located (SLURM cluster, the cluster of the virtual machines, the remote sites, etc). To meet the above requirements, HepJob was designed and developed. HepJob provides a set of simple user commands, for example: hep_sub, hep_q, hep_rm, etc. In the submission process, HepJob checks all the attributes and ensures all attributes are correct; assigns proper resources to users (the user and group info is obtained from the management database); routes jobs to the target site; performs other steps as required. Users can start with HepJob very easily and administrators can take the necessary management actions in HepJob. [ABSTRACT FROM AUTHOR]

Published
2020
Full Text
View/download PDF

Catalog

Books, media, physical & digital resources
See catalog results