Anuket Project
2022-11-23 Agenda and Minutes (11-16 meeting postponed)
- Al Morton
New Time: 6AM Pacific Wednesday. 3PM CET, 1PM GMT, 1930 India time US is on Standard Time. Pacific time will be UTC+0800.
Attendees
Al Morton Sridhar Rao Intern Shivank Chaudhary Trevor Cooper Luc Provoost Luc Provoost
| Item | Desc | who | Notes/minutes |
Special Topic: Containerizing PROX | Luc, Sridhar, Trevor, and all | Update Sridhar tested - same issue within container, start prox, many screen feedbacks - log file has been stored and shared. -Luc will look at this. Need to check DPDK, there may be something broken in DPDK? Luc will look at log file and nail down driver files, vswitch, maybe try testPMD instead of prox on the same container? - Sridhar will test this. If works, how many cores are used? -they are equally divided across the interfaces. Notes from Sridhar showed his message to Xavier. The main problem was a seg fault when trying to use two cores in a container Luc reports that he is using multiple cores everyday, he will share his docker file and other details. he has made some changes to prox as well. Sridhar says we are using virtual interfaces - Luc is using virtual pass-thru SR-IOV - he can't change this. Sridhar - we are using container to container testing - all virtual interfaces - VPP, User-space CNI https://github.com/intel/userspace-cni-network-plugin Luc - Try with his Docker File, also look into CPU policy, SR will check on policy. Luc will e-mail Docker file and Pod spec. Luc will look into this a bit, now that he is using containers. Luc - has problem with K8s when specifying Idle CPUs - assigns threads to single CPU and this is not wanted. - side issue. Xavier is using prox in new job at Redhat, so he is still active. He is also using containers. Prox is now maintained by Luc, with Xavier but not so accessible as in the past! Trevor - Question about Prox documentation, WHERE should we host the documentation now? Al - host on Anuket repo and GitHub Repo Mirror. Luc will copy repo to his personal. SampleVNF is a defunct project - what can we do about that? Future of Anuket is not certain. If we move to GH, how will permissions work? MIRROR would be the best of all solutions. THIS is the forum where we will track solutions, gerrit and GitHub, Patrice is an repo project expert, might be able to help with mirroring. SR- Prox should have been an independent project PROX - packet gen and execution - this is separable from SampleVNF Trevor's paper - summary: https://networkbuilders.intel.com/solutionslibrary/characterizing-telco-workloads-in-public-cloud-infrastructure#.Y2Jw7tpoaZk.link ANOTHER Project - OPI project using smart NICs. can use our tool and expertise, in the future. PROX - still the best solution in containerized generator context! Trex had lots of problems. AFA our experience is concerned. Please Sridhar summarize this in e-mail. SR can send with data also! Hood river - great views! Mountain biking! https://visithoodriver.com/ | |
Intern Update | Shivank |
Tim Gresham wants to know time horizon for the completion of the Intern project. Sridhar will try UNH - Sawyer or Lincoln Lavoie can get us started. Al sends mail with requirements eventually.... IXIA was loaned to LF, intel just hosting it. Maybe it can be moved to UNH! need ton investigate. Shivank: Status: Exams in progress. 3 tasks need to be completed. Where are we now?
Shivank's Internship page with progress: Internship 2022 - Benchmarking eBPF based solutions Internship 2022 - Benchmarking eBPF based solutions (pdf) If we can complete by end of the year - stay on Intel pod 12 Otherwise, Start to move to UNH LaaS Notes from Tasks assigned are complete except — need to change pod manifest files test setup with node 5, trex is not generating traffic Is cluster up? no do cleanup and do fresh installation - Ansible scripts are failing at the same stage. Need a call sometime and try to fix it together. k8sclustermanagement.yaml is not working. call scheduled to fix. Can we think of replacing CENTOS with Ubuntu? Yes we should do - CENTOS is very old...got to Ubuntu 22.04 LTS Server Edition. Take a backup and re-install. | |
| Discussion: Contribution on Containerized network benchmarking in BMWG session IETF-115 | Al | Slides: Considerations for Benchmarking Network Performance in Containerized Infrastructure https://datatracker.ietf.org/doc/draft-dcn-bmwg-containerized-infra/ yangun@dcn.ssu.ac.kr Sirdhar will contact. mipearlska1307@dcn.ssu.ac.kr presented at IETF-115 | |
Nile Progress | all | Review Sridhar Rao input Started Release Plan for ViNePerf Nile Sridhar has submitted 2 new patches, which will complete the dev schedule - Al must review. New SW versions and New OS versions. Adding Al as a reviewer. Using DPDK build with Meson and Ninja tools now. new tools. The build process is different for diff each OS now, diff folders for Fedora, etc. Need a discussion with Fulvio on the approach to use with building the CNIs. Sridhar will consider the questions to ask and try to exchange e-mail with Fulvio. Pod 12 is fine now! but we are the last user of the Intel community pods... The IXIA license has expired. Consider collaboration with Open Programmable Interface Project (Linux OPI) - need a member to join with us and start to identify common areas of interest. Sridhar will talk to Joe White, Dell TSC Chair. Node 3 in POD 13 has build issues - need to update that system. system dependencies. Build was successful in other locations, so we are interested to know where other projects are doing their builds. Maybe UNH Lab. | |
Sridhar presented at the Tech Discuss! | eBPF - final slides | ||
| eBPF topic (additional details) | Sridhar presented slides, which he used to familiarize Anuket with XDP and eBPF L3AF project in LFN - life cycle management on eBPF programs Cillium project has done some of their own benchmarking - but uses some unknown TCP: https://docs.cilium.io/en/stable/operations/performance/benchmark/ Not much uptake in telco - still many operations issues. The Cillium community agrees that they have not made a good case for telco usage (in conversations with Sridhar). | ||
| Shivank's Intern update |
Here are Shivank's updates for this week.
Shivank's CPU not supported by DPDK . Need this for most of our benchmarking tools New T-rex traffic Gen have been added and a new DPDK folder. Sirdhar shared https://github.com/intel/afxdp-plugins-for-kubernetes it is a possible option to use Shivank will investigate. Mail to Casey Cain about Intern compensation and feedback. Shivank's report for last week: https://github.com/Alt-Shivam/eBPF-CNI Pod 18 connectivity not working for several days, impacting Shivank's work. Tim G says working... Need Pod 12 working as well. There is an active JIRA ticket and comment stream. Shivank testing with a VM and K8s cluster - can try out locally Need to review mid-term Mentoring report - Al and Sridhar completed in last half of the meeting. ACTION Al Morton invite Maryam meeting to discuss performance (when we agree it's the right time). | ||
| Federica | Review Thesis proposal | ||
Post review of Daniele's work closure to Daniele's work and see if there is anything worth publishing, may consider less-selective conferences if novelty is less | Sridhar | Some issues we dig into now No reply from simonartxavier@gmail.com and Luc? Al and Sridhar sent new messages. Once we containerize prox, and we assign multiple cores to single interface, it crashes. how can we run Prox with multiple cores in a container?? | |
| Tasks and Action Plan for Shivank |
| ||
Open Programmable Interface Project (Linux OPI) Change CPU NIC architecture, more autonomy and CPU power goes to NIC in OPI, more than smart NIC now. New Trust boundary between Autonomous NIC and CPU. | Sridhar | New project, governing board formed and TSC meeting set. Opportunity for Benchmarking: work with us. Need to know planned NIC speeds, Need NIC HW to benchmark. There will be a test bed discussion, UNH and Keysight collaboration - using Keysight tools. We want to join the collaboration. Maybe UNH would welcome our help to some degree. Need some time to understand how the DUT HW will be obtained and installed. Also Remote access (for us, like Intel labs). | |
| Testbed - IXIA support | Sridhar/Al | 2nd ACTION: Do we have support from Ancuta or other IXIA person? Al Morton sent e-mail ACTION: License for IXIA HW - Check with Trevor Trevor Cooper says connected and powered on, but we still might have a license issue. Need Pierre's help. this activity seems to have stalled ... Pod 19 also not accessible - Dan Xu. ACTION: Al: can you add a comment in the Jira issue.. INFRA-7 ? As a PTL you can mention the dev is stuck with nodes being inaccessible. https://lf-anuket.atlassian.net/browse/INFRA-7 | |
| Additional Mentoring input | Sridhar/Federica | Federica Past meeting: python scripts. Possible combine 1 and 3 @Sridhar will expand on the proposal descriptions and Federica will begin to ask students about their interests. python scripting with K8s is possible for some students in 3rd year. Projects must have resume value, add skill set or new experience. Sridhar Rao created some projects. ACTION: See proposals & review https://wiki.anuket.io/display/HOME/Potential+Projects+for+Student+Volunteers | |
| Intern (background info) | Sridhar/Al/Federica/Shivank | XDP performance Studies for Cloud-Native NFV Use Cases "Maryam Tahhan joins the crew to talk XDP, AF_XDP, and fast networking" Operations support seems to be a big issue, and performance is currently about 0.7 of VPP and DPDK | |
| K8s on Pod 12 | Sridhar | will look into it |
Not DONE YET??? - Progress for Moselle (summary: items 5,6,and 7 lack the necessary automated address discovery feature, defer)
6/14 moved to 6/21 but no Moselle page update? Patches have been submitted to Releng, but have not been committed yet. Release manager must give go ahead after TSC approval.
Patches still open. possibly earlier, VinePerf Patches have been merged.
see Moselle Release
| 1 | Epic-VINEPERF-672:Create Tools as part of moselle release | 1. Done (Yet to submit the patch) 2. Another - Will not implement - Idea is already stale. | CCLA for Moselle release (LF as C) CCLA issue resolved Patch is submitted - Done. | |||
| 2 | Epic-VINEPERF-671:Add support for newer software versions | Completed testing. | Done | |||
| 3 | Epic-VINEPERF-670:Create clean workflows for Baremetal, Openstack and Kubernetes Usecases | Workflows are drawn-up. Implementation may need few more days. Request for 1 week more. | Minor work needed for K8s Workflow DONE??? I think so. | |||
| 4 | Epic-VINEPERF-669:Improve Stability for moselle Release | 1. Bug Fixes (2 patches) | Done | |||
| 5 | Epic-VINEPERF-652:Enhance XTesting-ViNePerf Integration | Moved to Next Release | depends on 7 | |||
| 6 | Task-VINEPERF-658:Enhance framework for XTesting-K8s Usecase | Partially done (reading results from output), Deployment tool. | 1 task remains | |||
| 7 | Task-VINEPERF-654:XTesting-ViNePerf Integration Enhancement - Kubernetes | Will not implement due to limitations with CNIs. Moved to Next release - if CNIs support this. | Need CNI to add flows automatically in Switches (Userspace-CNI, supports DPDK, OVS, VPP). Major impediment to integrate with X-Testing Sridhar will check with Xavier if ARP resolution is supported in Prox as a switch TBD | |||
| 8 | Task-VINEPERF-653:XTesting-ViNePerf Enhancement - Openstack [OLD] | Done | ||||
| 9 | Epic-VINEPERF-638:Dataplane performance testing for various internal (within cloud) scenarios | Completed - Yet to submit the patches - Done | Danielle will submit patches after Thesis on Friday, Monday 04/25 After review tomorrow, Daniele will submit the patch. Al Morton will need to review (big patch with documentation) - Done | |||
| 10 | Task-VINEPERF-643:Pod-Pod Communication | |||||