Anuket Project

Airship 1.8 (DPDK) FuncTest integration

Owned by James Gu
Last updated: Dec 21, 2020

The table below captures the current status of functest integration. Please remove the issue from the table when it is resolved,  


Cédric Ollivier

James Gu


Test suiteTest caseIssueResolution
Healthcheck


airship-opnfv-functest-healthcheck-latest-connection_check-run


airship-opnfv-functest-healthcheck-latest-tenantnetwork1-run


airship-opnfv-functest-healthcheck-latest-tenantnetwork2-run


airship-opnfv-functest-healthcheck-latest-vmready1-run


airship-opnfv-functest-healthcheck-latest-vmready2-run


airship-opnfv-functest-healthcheck-latest-singlevm1-run


airship-opnfv-functest-healthcheck-latest-singlevm2-run


airship-opnfv-functest-healthcheck-latest-vping_ssh-run


airship-opnfv-functest-healthcheck-latest-vping_userdata-run



vping_userdata-runUpstream bug: AIRSHIP-95. Nova instance console log api returns emptry
airship-opnfv-functest-healthcheck-latest-cinder_test-run


airship-opnfv-functest-healthcheck-latest-odl-run


airship-opnfv-functest-healthcheck-latest-tempest_smoke-run


airship-opnfv-functest-healthcheck-latest-tempest_horizon-run


Smoke


airship-opnfv-functest-smoke-latest-tempest_neutron-run



neutron_tempest_plugin.api.admin.test_dhcp_agent_scheduler.DHCPAgentSchedulersTestJSON.test_dhcp_port_status_activeRequires neutron extensions not supported in Airship DPDK

neutron_tempest_plugin.api.admin.test_dhcp_agent_scheduler.DHCPAgentSchedulersTestJSON.test_list_networks_hosted_by_one_dhcpsame 

neutron_tempest_plugin.api.admin.test_external_network_extension.ExternalNetworksRBACTestJSON.test_delete_policies_while_tenant_attached_to_netsame

neutron_tempest_plugin.api.admin.test_external_network_extension.ExternalNetworksRBACTestJSON.test_external_update_policy_from_wildcard_to_specific_tenantsame

neutron_tempest_plugin.api.admin.test_external_network_extension.ExternalNetworksRBACTestJSON.test_policy_allows_tenant_to_allocate_floatingipsame

neutron_tempest_plugin.api.admin.test_external_network_extension.ExternalNetworksRBACTestJSON.test_policy_allows_tenant_to_attach_ext_gwsame

neutron_tempest_plugin.api.test_dhcp_ipv6.NetworksTestDHCPv6.test_dhcp_stateful_fixedips_outrangesame

neutron_tempest_plugin.api.test_timestamp.TestTimeStampWithL3.test_create_floatingip_with_timestampsame

neutron_tempest_plugin.api.test_timestamp.TestTimeStampWithL3.test_create_router_with_timestampsame

neutron_tempest_plugin.api.test_timestamp.TestTimeStampWithL3.test_show_floatingip_attribute_with_timestampsame

neutron_tempest_plugin.api.test_timestamp.TestTimeStampWithL3.test_show_router_attribute_with_timestampsame

neutron_tempest_plugin.api.test_timestamp.TestTimeStampWithL3.test_update_floatingip_with_timestampsame

neutron_tempest_plugin.api.test_timestamp.TestTimeStampWithL3.test_update_router_with_timestampsame

QOS test casesUpstream: QOS support in DPDK

neutron_tempest_plugin.api.test_security_groups.RbacSharedSecurityGroupTest.test_regular_client_shares_to_another_regular_clientTBD
airship-opnfv-functest-smoke-latest-tempest_cinder-run



cinder_tempest_plugin.api.volume.test_volume_revert.VolumeRevertTests.test_volume_revert_to_snapshotAIRSHIP-96: upstream bug related to kernel 4.15
airship-opnfv-functest-smoke-latest-tempest_keystone-run


airship-opnfv-functest-smoke-latest-refstack_platform-run
skipped
airship-opnfv-functest-smoke-latest-tempest_full-run



tempest.api.compute.admin.test_instance_usage_audit_log_negative.InstanceUsageAuditLogNegativeTestJSON.test_instance_usage_audit_logs_with_nonadmin_user

AIRSHIP-104: Airship rbac policy inconsistent with tempest default

https://review.opendev.org/c/airship/treasuremap/+/767395

empest.api.compute.admin.test_live_migration.LiveAutoBlockMigrationV225Test.test_live_block_migration

1289064: upstream: live migration is broken in numa. Needs Train



tempest.api.compute.admin.test_live_migration.LiveAutoBlockMigrationV225Test.test_live_block_migration_paused

same



tempest.api.compute.admin.test_live_migration.LiveMigrationTest.test_live_block_migration

same



tempest.api.compute.admin.test_live_migration.LiveMigrationTest.test_live_block_migration_paused

same



tempest.api.compute.admin.test_migrations.MigrationsAdminTest.test_list_migrations_in_flavor_resize_situation

TBD



tempest.api.compute.admin.test_migrations.MigrationsAdminTest.test_resize_server_revert_deleted_flavor

TBD



tempest.api.compute.admin.test_simple_tenant_usage.TenantUsagesTestJSON.test_get_usage_tenant_with_non_admin_user

AIRSHIP-105: Airship rbac policy inconsistent with tempest default

https://review.opendev.org/c/airship/treasuremap/+/767395

tempest.api.compute.servers.test_attach_interfaces.AttachInterfacesTestJSON.test_create_list_show_delete_interfaces_by_network_port

fail 36.011



tempest.api.compute.servers.test_attach_interfaces.AttachInterfacesTestJSON.test_reassign_port_between_servers

fail 89.588



tempest.api.compute.servers.test_create_server_multi_nic.ServersTestMultiNic.test_verify_duplicate_network_nics

fail 126.883



tempest.api.compute.servers.test_create_server_multi_nic.ServersTestMultiNic.test_verify_multiple_nics_order

fail 125.284







tempest.api.compute.servers.test_delete_server.DeleteServersTestJSON.test_delete_server_while_in_verify_resize_state

TBD



tempest.api.compute.servers.test_device_tagging.TaggedAttachmentsTest.test_tagged_attachment

TBD



tempest.api.compute.servers.test_disk_config.ServerDiskConfigTestJSON.test_resize_server_from_auto_to_manual

TBD



tempest.api.compute.servers.test_disk_config.ServerDiskConfigTestJSON.test_resize_server_from_manual_to_auto

TBD



tempest.api.compute.servers.test_server_actions.ServerActionsTestJSON.test_get_console_output

Upstream bug: AIRSHIP-95. Nova instance console log api returns emptry



tempest.api.compute.servers.test_server_actions.ServerActionsTestJSON.test_get_console_output_server_id_in_shutoff_status

same



tempest.api.compute.servers.test_server_actions.ServerActionsTestJSON.test_get_console_output_with_unlimited_size

same



tempest.api.compute.servers.test_server_actions.ServerActionsTestJSON.test_resize_server_confirm

TBD



tempest.api.compute.servers.test_server_actions.ServerActionsTestJSON.test_resize_server_confirm_from_stopped

TBD



tempest.api.compute.servers.test_server_actions.ServerActionsTestJSON.test_resize_server_revert

TBD



tempest.api.compute.servers.test_server_actions.ServerActionsTestJSON.test_resize_server_revert_with_volume_attached

TBD



tempest.api.compute.servers.test_server_actions.ServerActionsTestJSON.test_resize_volume_backed_server_confirm

TBD



tempest.api.identity.admin.v3.test_application_credentials.ApplicationCredentialsV3AdminTest.test_create_application_credential_with_rolesAIRSHIP-106: Airship rbac policy inconsistent with tempest default

tempest.api.identity.admin.v3.test_groups.GroupsV3TestJSON.test_list_groupsAIRSHIP-107: possible rbac?

tempest.api.identity.admin.v3.test_list_users.UsersV3TestJSON.test_list_userszAIRSHIP-108: possible rbac

tempest.api.identity.admin.v3.test_list_users.UsersV3TestJSON.test_list_users_with_namesame

tempest.api.identity.admin.v3.test_projects_negative.ProjectsNegativeStaticTestJSON.test_list_projects_by_unauthorized_user

AIRSHIP-109: airship rbac policy is different than tempest default



tempest.api.identity.v3.test_application_credentials.ApplicationCredentialsV3Test.test_create_application_credentialAIRSHIP-110: airship rbac policy is different than tempest defaulthttps://review.opendev.org/c/airship/treasuremap/+/767395

tempest.api.identity.v3.test_application_credentials.ApplicationCredentialsV3Test.test_create_application_credential_expiressame

tempest.api.identity.v3.test_application_credentials.ApplicationCredentialsV3Test.test_list_application_credentialssame

tempest.api.identity.v3.test_application_credentials.ApplicationCredentialsV3Test.test_query_application_credentialssame

tempest.api.identity.v3.test_users.IdentityV3UsersTest.test_password_history_check_self_service_api

TBD: possible rbac



tempest.api.identity.v3.test_users.IdentityV3UsersTest.test_user_account_lockout

TBD: possible rbac



tempest.api.identity.v3.test_users.IdentityV3UsersTest.test_user_update_own_password

TBD: possible rbac



tempest.api.image.v2.test_images.BasicOperationsImagesTest.test_deactivate_reactivate_imageTBD: possible rbac

tempest.api.network.admin.test_dhcp_agent_scheduler.DHCPAgentSchedulersTestJSON.test_list_networks_hosted_by_one_dhcpFUNCTEST-1023: L3 test cases need to be skipped based on Neutron extension list or blacklisted

tempest.api.network.admin.test_external_network_extension.ExternalNetworksTestJSON.test_list_external_networkssame

tempest.api.network.admin.test_external_network_extension.ExternalNetworksTestJSON.test_show_external_networks_attributesame

tempest.api.network.test_dhcp_ipv6.NetworksTestDHCPv6.test_dhcp_statefulsame

tempest.api.network.test_dhcp_ipv6.NetworksTestDHCPv6.test_dhcp_stateful_fixedipssame

tempest.api.network.test_dhcp_ipv6.NetworksTestDHCPv6.test_dhcp_stateful_fixedips_duplicatesame

tempest.api.network.test_dhcp_ipv6.NetworksTestDHCPv6.test_dhcp_stateful_fixedips_outrangesame

tempest.api.network.test_dhcp_ipv6.NetworksTestDHCPv6.test_dhcp_stateful_routersame

tempest.api.network.test_dhcp_ipv6.NetworksTestDHCPv6.test_dhcpv6_64_subnetssame

tempest.api.network.test_dhcp_ipv6.NetworksTestDHCPv6.test_dhcpv6_invalid_optionssame

tempest.api.network.test_dhcp_ipv6.NetworksTestDHCPv6.test_dhcpv6_stateless_eui64same

tempest.api.network.test_dhcp_ipv6.NetworksTestDHCPv6.test_dhcpv6_stateless_no_rasame

tempest.api.network.test_dhcp_ipv6.NetworksTestDHCPv6.test_dhcpv6_stateless_no_ra_no_dhcpsame

tempest.api.network.test_dhcp_ipv6.NetworksTestDHCPv6.test_dhcpv6_two_subnetssame

tempest.api.network.test_ports.PortsIpV6TestJSON.test_port_list_filter_by_router_idsame

tempest.api.network.test_ports.PortsTestJSON.test_port_list_filter_by_router_idsame

tempest.api.volume.admin.test_volume_manage.VolumeManageAdminTest.test_unmanage_manage_volume

TBD



tempest.api.volume.test_volumes_negative.VolumesNegativeTest.test_create_volume_from_deactivated_image

AIRSHIP-112: possible rbac?














airship-opnfv-functest-smoke-latest-tempest_heat-run



heat_tempest_plugin.tests.functional.test_nova_server_networks.CreateServerTest.test_create_server_with_portFUNCTEST_1022: Requires neutron l3 extension not supported in Airship DPDK. Should be dynamically skipped based on neutron extension list.

heat_tempest_plugin.tests.functional.test_nova_server_networks.CreateServerTest.test_create_server_with_subnet_fixed_ip_sec_groupsame

heat_tempest_plugin.tests.functional.test_nova_server_networks.CreateServerTest.test_create_update_server_with_subnetsame

heat_tempest_plugin.tests.functional.test_nova_server_networks.UpdateServerNetworksTest.test_create_update_server_add_networksame

heat_tempest_plugin.tests.functional.test_nova_server_networks.UpdateServerNetworksTest.test_create_update_server_add_same_fixed_ipsame

heat_tempest_plugin.tests.functional.test_nova_server_networks.UpdateServerNetworksTest.test_create_update_server_add_subnetsame

heat_tempest_plugin.tests.functional.test_nova_server_networks.UpdateServerNetworksTest.test_create_update_server_multi_networks_swapssame

heat_tempest_plugin.tests.functional.test_nova_server_networks.UpdateServerNetworksTest.test_create_update_server_swap_network_portsame

heat_tempest_plugin.tests.functional.test_nova_server_networks.UpdateServerNetworksTest.test_create_update_server_swap_network_subnetsame

heat_tempest_plugin.tests.functional.test_nova_server_networks.UpdateServerNetworksTest.test_create_update_server_swap_subnet_networksame

heat_tempest_plugin.tests.scenario.test_server_signal.ServerSignalIntegrationTest.test_server_signal_userdata_format_rawsame

heat_tempest_plugin.tests.scenario.test_server_signal.ServerSignalIntegrationTest.test_server_signal_userdata_format_software_configsame

heat_tempest_plugin.tests.functional.test_os_wait_condition.OSWaitCondition.test_create_stack_with_multi_signal_waitconditionTBD

heat_tempest_plugin.tests.scenario.test_base_resources.BasicResourcesTest.test_base_resources_integrationTBD

heat_tempest_plugin.tests.scenario.test_remote_deeply_nested.RemoteDeeplyNestedStackTest.test_remote_nestedTBD
airship-opnfv-functest-smoke-latest-tempest_scenario-run



tempest.scenario.test_server_basic_ops.TestServerBasicOps.test_server_basic_opsUpstream bug: AIRSHIP-103. Nova metaservice url not reachable from guest vm
airship-opnfv-functest-smoke-latest-tempest_telemetry-run
skipped
airship-opnfv-functest-smoke-latest-tempest_slow-run



tempest.scenario.test_volume_backup_restore.TestVolumeBackupRestore.test_volume_backup_restoreFUNCTEST-1019: teste volume backup depends on floating ip

tempest.api.compute.servers.test_device_tagging.TaggedBootDevicesTest.test_tagged_boot_devicesFUNCTEST-1020: test device tagging still uses tenant network
airship-opnfv-functest-smoke-latest-rally_sanity-run



NovaServers.boot_and_live_migrate_serverNo support from OpenStack Helm. Recommend to blacklist.
airship-opnfv-functest-smoke-latest-patrole-run



patrole_tempest_plugin.tests.api.network.test_floating_ips_rbac.FloatingIpsRbacTest.test_create_floating_ipFUNCTEST-1021: Requires neutron l3 extension not supported in Airship DPDK. Should be dynamically skipped based on neutron extension list.

patrole_tempest_plugin.tests.api.network.test_floating_ips_rbac.FloatingIpsRbacTest.test_create_floating_ip_floatingip_addresssame

patrole_tempest_plugin.tests.api.network.test_floating_ips_rbac.FloatingIpsRbacTest.test_delete_floating_ipsame

patrole_tempest_plugin.tests.api.network.test_floating_ips_rbac.FloatingIpsRbacTest.test_list_floating_ipssame

patrole_tempest_plugin.tests.api.network.test_floating_ips_rbac.FloatingIpsRbacTest.test_show_floating_ipsame

patrole_tempest_plugin.tests.api.network.test_floating_ips_rbac.FloatingIpsRbacTest.test_update_floating_ipsame

patrole_tempest_plugin.tests.api.network.test_networks_rbac.NetworksRbacTest.test_create_network_is_defaultsame
airship-opnfv-functest-smoke-latest-refstack_compute-run
skipped
airship-opnfv-functest-smoke-latest-refstack_object-run


airship-opnfv-functest-smoke-latest-networking-sfc-run
skipped
airship-opnfv-functest-smoke-latest-networking-bgpvpn-run
skipped
airship-opnfv-functest-smoke-latest-tempest_barbican-run
skipped
airship-opnfv-functest-smoke-latest-tempest_octavia-run
skipped
Benchmark


airship-opnfv-functest-benchmarking-latest-rally_full-run



NeutronNetworks.create_and_show_ports

needs blacklist. It needs router extension.



NovaServers.resize_serverSLA. Retest on pod 15. pod17 too small.

GlanceImages.create_and_deactivate_imageSLA. Retest on pod 15. pod17 too small.
airship-opnfv-functest-benchmarking-latest-rally_jobs-run
skipped
airship-opnfv-functest-benchmarking-latest-vmtp-run
skipped
airship-opnfv-functest-benchmarking-latest-shaker-run
skipped
VNF


airship-opnfv-functest-vnf-latest-cloudify-run
FUNCTEST-1017: cloudify vpn deploy uses floating ip. Shoudl change to use "public" network or skip.
airship-opnfv-functest-vnf-latest-cloudify_ims-run
skipped
airship-opnfv-functest-vnf-latest-heat_ims-run
skipped
airship-opnfv-functest-vnf-latest-vyos_vrouter-run
skipped
airship-opnfv-functest-vnf-latest-juju_epc-run
skipped