Subject: Re: [nmrg] [External] call for adoption as RG document : draft-zhou-nmrg-digitaltwin-network-concepts

From: "Cheng Zhou" <[email-masked]>
Date: Tue, 4 Jan 2022 21:57:27 +0800

Hi Daniel, 

Thank you very much for the valuable comments. 

Please check our initial reply inline. We will update the draft later after in-depth study on your comments.

Best Regards,

Cheng Zhou 

发件人: nmrg [mailto:[email-masked]] 代表 King, Daniel
发送时间: 2021年12月25日 08:08
收件人: [email-masked]
抄送: [email-masked]
主题: Re: [nmrg] [External] call for adoption as RG document : draft-zhou-nmrg-digitaltwin-network-concepts

Hi RG, 

Having read the draft, I think it's a valuable work item for the RG, and I support the adoption of the I-D. However, significant work by the authors and input from the RG will be required, as expected with a recently adopted document. 

In response to the questions posed:

Q1.  NMRG being a *research* group, it is important to articulate well the problem(s), challenge(s) and research questions.

Yes, but. The document would benefit from several significant updates that are proposed in more detail below. 

Q2. Is the scientific soundness of the draft satisfying? (quality of the scientific proposition, rigor of the method, impact) ; "the measure of success is the quality and impact of the research results" [RFC2014] 

Within reason. The document is informational and provides an overview of the digital twin network state of the art. It would benefit from resequencing sections and more academic research papers/journals that also cite common challenges. The authors discuss a broad set of use cases. It may be better to focus on one or two initially and go deeper in the discussion.

<Authors> Accept your suggestion. We will revise the draft in next version via resequencing sections and citing more research papers. And, we will consider digging deeper into one or two cases.

3.  As the document proposes an *architecture*, it is important to provide a clear scope and positioning of this architecture with respect to network management/automation/operation frameworks and where/how the proposed architecture will prove useful or aims at replacing/augmenting certain functionality. 

I think the document's readability would be improved if the authors made the core requirements and research challenge more evident/earlier in the document, perhaps as research questions raised by the authors and citing other research (a few links provided below) that highlights challenges. Then when the authors present the architecture and functional components, they could may how it solves the specific challenges outlined earlier in the document. 

<Authors> Accept your suggestion. We will cite more researches, and resequece the sections in next version of draft. 

Please find my specific comments and numerous editorial suggestions and questions below.

1. Introduction 

Replace: "The fast growing of network scale and the increased demand placed on these networks, requires them to accommodate and adapt dynamically to customer needs, implying a big challenge to network operators."

With: "The fast growth of network scale and the increased demand placed on these networks require them to accommodate and adapt dynamically to customer needs, implying a significant challenge to network operators."

Replace: "Indeed, network operation and maintenance are becoming more complex due to the higher complexity of the managed networks and their sophisticated services."

With: "Indeed, network operation and maintenance are becoming more complex due to the higher complexity of the managed networks and their sophisticated services."

Replace: "full lifecycle and full business data-driven."

With: "full lifecycle and comprehensive business data-driven."

Replace: "The main difference compared to traditional network management system is the use of interactive virtual-real mapping to build closed-loop network automation."

With: "The main difference compared to traditional network management systems is interactive virtual-real mapping to build closed-loop network automation."

Replace: " Having an emulation platform that allows to reliably represent the state of a network is more dependable than a simulation platform."

With: " An emulation platform that highlights the state of network reliability provides a more helpful simulation platform."

Replace: " Service impact analysis tasks will also be facilitated."

With: " In addition, service impact analysis tasks will also be facilitated."

At this point, I just exported the text I-D into an MS Word document, enabled change markers and edited directly. I will provide this to the authors as an MS Word file. 

<Authors> Thanks for the comments. Appreciating and waiting for your MS Word file.  

2. Requirements Language 

As an informational document, do you need to use specific requirement language? If you intend to have hard and soft requirements in future discussions, it may make more sense to keep this section. 

<Authors> We will remove the ‘Requirements Language’ section in next version. 

3. Definitions and Acronyms

This section feels a little light given the various technology areas you discuss. Also, it only provides expanded acronym definitions, and I would think some definitions such as "digital twin" would also be helpful. 

<Authors> Accept your suggestion. We will add some more definitions in next version.

4. Definition of Digital Twin Network

The concept of "digital twin" predates 2003 (Michael Grieves). My understanding is "digital twin" was initially coined by NASA in the 1970s during the Apollo 13 program so that terrestrial engineers could model the extra-terrestrial effects on  Apollo 13 objects and systems. You might want to update your etymology and add a citation, such as "S. Ferguson, "Apollo 13: The First Digital Twin", Article by Siemens Blogs, April 14, 2020. https://blogs.sw.siemens.com/simcenter/apollo-13-the-first-digital-twin/"

<Authors> Thanks for the information. From more papers, it is more widely accepted that ‘digital twin’ concept was initially introduced in 2003 by Michael Grieves. 

So, we prefer to keep current description on the origin of ‘digital twin’. 

Before you define "digital twin network", you might want to enlighten the readers about the "digital twin" concept. This might be a definition in section 3., or a paragraph in section 5 along the lines of:

"A digital twin is a virtual representation of the real-time digital counterpart, including physical objects, systems or processes. A digital twin is generated by gathering data and simulating the environment so that machine learning reasoning may be applied, often the entire lifecycle of the object or system is also represented."

<< 

<Authors> Accept your suggestion. We will add definition of ‘digital twin’ in section 3.

It may be useful to highlight the differences between a digital "model", "shadow" or " clone" of a system, and "digital twin". A key difference will be the direction of data between the physical and virtual systems. For instance, you might state:

"Typically, when using a digital twin the system is generated and then synchronized using data flows in both directions between physical and digital components, so that control data can be sent and changes between the physical and digital objectives and systems are automatically represented. This behavior is unlike a digital model, shadow or clone, which are synchronized manually,  lack control data, and will not have a full cycle of data integrated." 

<< 

 <Authors> Thanks for the information. We will study in-depth then consider adding the info in next version. 

This section then launches into the discussion of the "digital twin network". It proposes "Key Elements of Digital Twin Network" which on my initial readthrough, read like functional components, which then leads into architecture later in the document. It may be better to reorganise the document for the reader to clearly state the general problem scope, then research challenges and requirements not just in this document but also from other research papers. Then you can highlight your architecture and functional components as a way to address those research challenges. 

<Authors> Thanks for the suggestion. We will check and discuss internally; then cite more researches, and resequece the sections in next version of draft.  

Several journals refer to digital twin research challenges for specific vertical industries (with a focus on manufacturing and healthcare), I appreciate this document is focused on the telco network research, but it may be worth highlighting other research efforts as there seem to be several common challenges across technology areas. In addition, this document may benefit from a specific section on telco networking use cases, including smart cities and next generation infrastructure. Then you can summarise some of the more pertinent research challenges outlined in several documents I found relevant:

 <Authors> Thanks for the valuable information.  We will study in-depth and cite more info related on applying ‘digital twin’ in networking field. 

ITU-T Representative use cases and key network requirements for Network 2030

https://www.itu.int/en/ITU-T/focusgroups/net2030/Documents/Technical_Report.pdf

Digital Twin: Enabling Technologies, Challenges and Open Research

https://ieeexplore.ieee.org/abstract/document/9103025

Digital Twin: Values, Challenges and Enablers

https://arxiv.org/abs/1910.01719

Connecting the Twins: A Review on Digital Twin Technology & its Networking Requirements

https://www.sciencedirect.com/science/article/pii/S1877050921006694

Digital Twin: Values, Challenges and Enablers From a Modeling Perspective

https://ieeexplore.ieee.org/document/8972429

Digital Twin Technologies and Smart Cities

https://citations.springernature.com/book?doi=10.1007/978-3-030-18732-3

Smart Cities with Digital Twin Systems for Disaster Management

https://ascelibrary.org/doi/full/10.1061/%28ASCE%29ME.1943-5479.0000779

Realizing the Digital Twin Transition for Smart Cities

https://portal.dnb.de/opac.htm?method=simpleSearch <https://portal.dnb.de/opac.htm?method=simpleSearch&cqlMode=true&query=idn%3D1240130449> &cqlMode=true&query=idn%3D1240130449

The Convergence of Digital Twin, IoT, and Machine Learning: Transforming Data into Action

https://link.springer.com/chapter/10.1007/978-3-030-18732-3_1

The Role of Digital Twin in Optical Communication: Fault Management, Hardware Configuration, and Transmission Simulation

https://ieeexplore.ieee.org/abstract/document/9356524

Digital Twin for 5G and Beyond

https://ieeexplore.ieee.org/abstract/document/9374645

Digital Twin in Industry 4.0: Technologies, Applications and Challenges

https://ieeexplore.ieee.org/abstract/document/8972134

Have you considered a section/sub-section on developing standards (use cases, requirements and architecture) efforts for digital twin network technologies? 

Digital Twin Manufacturing Framework: https://www.iso.org/standard/75066.html

Focus Group on Technologies for Network 2030 (FG NET-2030): https://www.itu.int/en/ITU-T/focusgroups/net2030/Documents/Technical_Report.pdf

Joint Advisory Group (JAG) Group on Emerging Technology and Innovation (JETI) https://jtc1info.org/technology/advisory-groups/jeti/

 <Authors> The listed ‘standard works’ are most of ‘digital twin in general or vertical industry’ instead ‘digital twin in network field’. Our initial opinion is to describe it in short as the background of digital twin.

Several recent technologies discussed in the IRTF/IETF are applicable digital twin networks. I think the document would benefit from a section on "Enabling Technologies" to support the proposed architecture and functional components. These advanced/recent technologies are related to:

- Network connectivity 

- Real-time data communication, ultra-low latency and time-based networking 

- Collaboration management, conflict detection and resolution 

- Data privacy and integrity, trust and security 

 <Authors> The draft is focusing on concept and architecture of digital twin network, not including enabling technologies. Actually, we believe that each ‘enabling technology’ is worth of a separate draft to study in details in future. We will check internally whether to add a section to describe the enabling technologies in brief. 

Indeed, several exciting research challenges exist for digital twin networking. Still, you might also document/cite recent implementations and highlight deficiencies, and in some cases, gaps are highlighted in the implementation documentation, such as:

Digital Twins for IoT Applications: A Comprehensive Approach to Implementing IoT Digital Twins

https://www.semanticscholar.org/paper/A-Comprehensive-Approach-to-Implementing-IoT-Twins/5d0482cf23a52da8cf7bf8be05e1e483dc296bf0

 <Authors> Thanks for the information. We will study in-depth and citing more docs/papers in next version. 

I hope these thoughts and suggestions are helpful. 

BR, Dan.

From: nmrg <[email-masked]> On Behalf Of Ciavaglia, Laurent
Sent: 10 December 2021 12:02
To: [email-masked]
Cc: [email-masked]
Subject: [External] [nmrg] call for adoption as RG document : draft-zhou-nmrg-digitaltwin-network-concepts

This email originated outside the University. Check before clicking links or attachments.

Dear NMRG,

This message starts the call for adoption as RG document for:

Digital Twin Network: Concepts and Reference Architecture - draft-zhou-nmrg-digitaltwin-network-concepts

https://datatracker.ietf.org/doc/draft-zhou-nmrg-digitaltwin-network-concepts/ <https://eur02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdatatracker.ietf.org%2Fdoc%2Fdraft-zhou-nmrg-digitaltwin-network-concepts%2F&data=04%7C01%7Ckingd%40live.lancs.ac.uk%7C52f2f1ce1a56405824ba08d9bbd4e2aa%7C9c9bcd11977a4e9ca9a0bc734090164a%7C0%7C0%7C637747346297798837%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C2000&sdata=BVMByqjHFN0OKdWT3G5M4RrGL%2BaPZ1Bxorxn23xJ0hQ%3D&reserved=0> 

Please let us know if you support or not the work becoming a RG document.

In all cases, provide detailed comments to support your opinion and send them on the mailing list.

We are particularly interested in assessing the following aspects:

1.      NMRG being a *research* group it is important to articulate well the problem(s), challenge(s) and research questions. 

2.      Is the scientific soundness of the draft satisfying? (quality of the scientific proposition, rigor of the method, impact) ; “the measure of success is the quality and impact of the research results” [RFC2014]

3.      As the document proposes an *architecture*, it is important to provide a clear scope and positioning of this architecture with respect to network management/automation/operation frameworks and where/how the proposed architecture will prove useful or aims at replacing/augmenting certain functionality

The procedure for RG document adoption and important criteria are detailed here: https://mailarchive.ietf.org/arch/msg/nmrg/CVEyLUvfxJk1Ud5WdM9Y5LGvQmU <https://eur02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmailarchive.ietf.org%2Farch%2Fmsg%2Fnmrg%2FCVEyLUvfxJk1Ud5WdM9Y5LGvQmU&data=04%7C01%7Ckingd%40live.lancs.ac.uk%7C52f2f1ce1a56405824ba08d9bbd4e2aa%7C9c9bcd11977a4e9ca9a0bc734090164a%7C0%7C0%7C637747346297798837%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C2000&sdata=Wz5A%2BUG2BTW97t746YhlsagW0fKebZUDtxyoLubkRuc%3D&reserved=0> 

Given the time of the year, this call for adoption is open for four weeks and ends on January 7th, 2022.

Best regards, NMRG chairs

Laurent & Jérôme

---

From: Jordi Paillissé <[email-masked]>
Date: Wed, 5 Jan 2022 13:51:58 +0100

Dear NMRG and Laurent,

I have read and support the adoption of the draft as a RG document.

I believe it addresses a relevant problem and outlines the next step in 
network modeling, that can help in managing large-scale networks. There 
are articles in the literature that discuss the limitations of current 
modeling tools [1].

I think that the draft needs some more level of detail but I’m fine 
discussing after adoption, such as:
     • Which are the outputs of the digital twin? Eg. I was thinking of 
performance metrics such as delay or loss, but the authors may have 
others in mind.
     • I would emphasize the use case of network optimization (it is 
already mentioned in sec. 5), but I’d add that the applications 
connected to the digital twin can be network optimizers that find 
optimal configurations. Since you can connect different applications, 
you can try different optimization techniques / algorithms.
     • I also agree with Daniel that the draft needs more references.

Thanks, and happy new year!

Jordi

[1] Qizhen Zhang, Kelvin K. W. Ng, Charles Kazer, Shen Yan, João Sedoc, 
and Vincent Liu. 2021. MimicNet: fast performance estimates for data 
center networks with machine learning. In Proceedings of the 2021 ACM 
SIGCOMM 2021 Conference (SIGCOMM '21). Association for Computing 
Machinery, New York, NY, USA, 287–304. 
DOI:https://doi.org/10.1145/3452296.3472926

El 10/12/21 a les 13:01, Ciavaglia, Laurent ha escrit:
> Digital Twin Network: Concepts and Reference Architecture - 
> draft-zhou-nmrg-digitaltwin-network-concepts
>
> https://datatracker.ietf.org/doc/draft-zhou-nmrg-digitaltwin-network-concepts/
>
> *In all cases, provide detailed comments to support your opinion and 
> send them on the mailing list.*
>
>  1. NMRG being a *research* group it is important to articulate well
>     the problem(s), challenge(s) and research questions.
>  2. Is the scientific soundness of the draft satisfying? (quality of
>     the scientific proposition, rigor of the method, impact) ; “the
>     measure of success is the quality and impact of the research
>     results” [RFC2014]
>  3. As the document proposes an *architecture*, it is important to
>     provide a clear scope and positioning of this architecture with
>     respect to network management/automation/operation frameworks and
>     where/how the proposed architecture will prove useful or aims at
>     replacing/augmenting certain functionality
>
> The procedure for RG document adoption and important criteria are 
> detailed here: 
> https://mailarchive.ietf.org/arch/msg/nmrg/CVEyLUvfxJk1Ud5WdM9Y5LGvQmU
>
> Given the time of the year, this call for adoption is open for four 
> weeks and *ends on January 7^th , 2022*.
>
> _______________________________________________
> nmrg mailing list
> [email-masked]
> https://www.irtf.org/mailman/listinfo/nmrg

---

From: Jérôme François <[email-masked]>
Date: Fri, 7 Jan 2022 23:16:41 +0100

Dear all,

Digital Twin (DT) is gaining interest in the community and so important to 
consider in NMRG. As NMRG participant, I think so this document is valuable for 
the group. The proposed architecture is generic enough to let additional 
refinement regarding the different models to be proposed afterwards (by others). 
It can help bootstrapping this topic in NMRG. I support its adoption.

However, in my opinion, it will require several enhancements with the help of the 
group. The content is unbalanced with under-represented  research aspects. In 
particular, challenges in section 7 could be more described by positioning these 
challenges in regards to current technologies and their lacks. For example, one 
challenge is the scalability of data acquisition. There are many solutions for 
large-scale monitoring in network and data pipelines and it is unclear how this 
challenge (and their solutions) differs in the context of DT.

Below you can find more general comments. More specific comments have been added 
in-line in the attached document.

1) the document could better emphasize the  benefit of a DT in case of network 
(management) in comparison to other platforms. For instance, the value of DTs is 
clear for physical or cyber-physical systems such as industry systems/networks. In 
case of objects which are already massively digital like networks, the benefits 
and challenges could be different. Highlighting that would help to support that 
research dedicated to DT for network is necessary.

2) Once the need for DTs specific to network is well motivated and justified, it 
will be then easier to highlight the different challenges and also explain how the 
proposed architecture addresses these challenges.

3) Motivate the need for a new framework or architecture in comparison to 
emulation platform or cyber-range (cited in the context of security training). One 
value could be the real-time interaction between the twins. However, when reading 
the document, (especially the proposed applications), my understanding is that 
interactions are unidirectional and one-shot: you clone the real twin and its 
state to a digital twin. Once done, you use the DT for doing some tests 
independently of the real twin ? (If you do some tests on the digital twin, the 
data produced  by the real system cannot reflect the state of the DT.

Best regards
jerome

Le 10/12/2021 à 13:01, Ciavaglia, Laurent a écrit :
> <https://datatracker.ietf.org/doc/draft-zhou-nmrg-digitaltwin-network-concepts/>
>
> *In all cases, provide detailed comments to support your opinion and send them 
> on the mailing list.*
>
>  1. NMRG being a *research* group it is important to articulate well the
>     problem(s), challenge(s) and research questions.
>  2. Is the scientific soundness of the draft satisfying? (quality of the
>     scientific proposition, rigor of the method, impact) ; “the measure of
>     success is the quality and impact of the research results” [RFC2014]
>  3. As the document proposes an *architecture*, it is important to provide a
>     clear scope and positioning of this architecture with respect to network
>     management/automation/operation frameworks and where/how the proposed
>     architecture will prove useful or aims at replacing/augmenting certain
>     functionality
>
> The procedure for RG document adoption and important criteria are detailed here: 
> <https://mailarchive.ietf.org/arch/msg/nmrg/CVEyLUvfxJk1Ud5WdM9Y5LGvQmU>
>
> Given the time of the year, this call for adoption is open for four weeks and 
> *ends on January 7^th , 2022*.
>

---

From: "Cheng Zhou" <[email-masked]>
Date: Tue, 11 Jan 2022 15:07:27 +0800

Hi Jerome, 

Please check  my reply inline, as well as the reply in attached doc. 

Thank and Best Regards,

Cheng Zhou

发件人: nmrg [mailto:[email-masked]] 代表 Jér?me Fran?ois
发送时间: 2022年1月8日 06:17
收件人: Ciavaglia, Laurent; [email-masked]
抄送: [email-masked]
主题: Re: [nmrg] call for adoption as RG document : draft-zhou-nmrg-digitaltwin-network-concepts

Dear all,

Digital Twin (DT) is gaining interest in the community and so important to consider in NMRG. As NMRG participant, I think so this document is valuable for the group. The proposed architecture is generic enough to let additional refinement regarding the different models to be proposed afterwards (by others). It can help bootstrapping this topic in NMRG. I support its adoption.

However, in my opinion, it will require several enhancements with the help of the group. The content is unbalanced with under-represented  research aspects. In particular, challenges in section 7 could be more described by positioning these challenges in regards to current technologies and their lacks. For example, one challenge is the scalability of data acquisition. There are many solutions for large-scale monitoring in network and data pipelines and it is unclear how this challenge (and their solutions) differs in the context of DT.
<Cheng Zhou>  Accept your suggestion, and we will refine section 7 via describing more enabling technologies that can address the challenges. 

Below you can find more general comments. More specific comments have been added in-line in the attached document.
<Cheng Zhou>  Really appreciate your detailed comments. Attached contains my replies to all your comments in revision mode.

1) the document could better emphasize the  benefit of a DT in case of network (management) in comparison to other platforms. For instance, the value of DTs is clear for physical or cyber-physical systems such as industry systems/networks. In case of objects which are already massively digital like networks, the benefits and challenges could be different. Highlighting that would help to support that research dedicated to DT for network is necessary.

<Cheng Zhou>  Accept your suggestion. Copmared with classical approaches, key benefits of DT network can be summarized as below:

1) Using real-time data to establish high fidelity twins, the effectiveness of network simulation is higher; then the simulation cost will be relatively low. 

2) The impact and risk on running network is low when automatically applying config/policy changes after full analysis and verification on DT. 

3) The faults of the physical network can be automatically captured via capturing and analyzing real-time data, then the correction strategy can be distributed to the physical network after analysis in the twins to complete the closed-loop automatic fault repair.

We will refine section 5 in next version via emphasizing the comparison.  

2) Once the need for DTs specific to network is well motivated and justified, it will be then easier to highlight the different challenges and also explain how the proposed architecture addresses these challenges.
<Cheng Zhou>  Accept your suggestion, and will refine in next version.

3) Motivate the need for a new framework or architecture in comparison to emulation platform or cyber-range (cited in the context of security training). One value could be the real-time interaction between the twins. However, when reading the document, (especially the proposed applications), my understanding is that interactions are unidirectional and one-shot: you clone the real twin and its state to a digital twin. Once done, you use the DT for doing some tests independently of the real twin ? (If you do some tests on the digital twin, the data produced  by the real system cannot reflect the state of the DT.
<Cheng Zhou> Thanks for the comment.  In case just doing some test in digital twin without interfering the real twin, the interactions can be unidirectional, but not one-shot. While the testing, digital twin can also get data refreshed from real twin, then the proposed test can be better validated to see whether it can accommodate with network changes. 

In most other cases (e.g. network optimization, network fault recovery, etc.), the virtual-real interactions between the twins are bidirectional and also persistent. Real data should continuously reported or trigger-reported to digital twin,  then digital twin can automatically monitor and analyze the network state based on the running data. Whenever changes are needed, digital twin will deliver the changes to real twin via the control interfaces. 

Best regards
jerome 

Dear NMRG,

The procedure for RG document adoption and important criteria are detailed here: https://mailarchive.ietf.org/arch/msg/nmrg/CVEyLUvfxJk1Ud5WdM9Y5LGvQmU

Laurent & Jérôme

---

From: "Ciavaglia, Laurent" <[email-masked]>
Date: Fri, 14 Jan 2022 18:20:59 +0000

Dear NMRG,

The call for adoption of draft-zhou-nmrg-digitaltwin-network-concepts has ended on Friday, January 7th.

The RG chairs are reviewing the comments received and will communicate shortly on the next steps.

Best regards, Jérôme and Laurent

From: Ciavaglia, Laurent
Sent: Friday, December 10, 2021 01:02 PM
To: [email-masked]
Cc: [email-masked]
Subject: call for adoption as RG document : draft-zhou-nmrg-digitaltwin-network-concepts

Dear NMRG,

  1.  NMRG being a *research* group it is important to articulate well the problem(s), challenge(s) and research questions.
  2.  Is the scientific soundness of the draft satisfying? (quality of the scientific proposition, rigor of the method, impact) ; "the measure of success is the quality and impact of the research results" [RFC2014]
  3.  As the document proposes an *architecture*, it is important to provide a clear scope and positioning of this architecture with respect to network management/automation/operation frameworks and where/how the proposed architecture will prove useful or aims at replacing/augmenting certain functionality