Versions Compared

Old Version 5

changes.mady.by.user James Gu

Saved on

compared with

New Version 6

changes.mady.by.user James Gu

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

The table below captures the current functest integration status.

Participants:

Cédric Ollivier

James Gu

...

The table below captures the current functest integration status.


Participants:

Cédric Ollivier

James Gu


Test suiteTest caseStatus
Healthcheck

airship-opnfv-functest-healthcheck-latest-
vmready2
connection_check-run

airship-opnfv-functest-healthcheck-latest-
singlevm1
tenantnetwork1-run

airship-opnfv-functest-healthcheck-latest-
singlevm2
tenantnetwork2-run

airship-opnfv-functest-healthcheck-latest-
vping_ssh
vmready1-run

airship-opnfv-functest-healthcheck-latest-
vping_
vmready2-run

airship-opnfv-functest-healthcheck-latest-singlevm1-run

airship-opnfv-functest-healthcheck-latest-singlevm2-run

airship-opnfv-functest-healthcheck-latest-vping_ssh-run

airship-opnfv-functest-healthcheck-latest-vping_userdata-run


vping_userdata-runUpstream bug: AIRSHIP-95. Nova instance console log api returns emptry
airship-opnfv-functest-healthcheck-latest-cinder_test-run

airship-opnfv-functest-healthcheck-latest-odl-run

airship-opnfv-functest-healthcheck-latest-tempest_smoke-run

airship-opnfv-functest-healthcheck-latest-tempest_horizon-run

Smoke

airship-opnfv-functest-smoke-latest-tempest_neutron-run


neutron_tempest_plugin.api.admin.test_dhcp_agent_scheduler.DHCPAgentSchedulersTestJSON.test_dhcp_port_status_activeRequires neutron extensions not supported in Airship DPDK

neutron_tempest_plugin.api.admin.test_dhcp_agent_scheduler.DHCPAgentSchedulersTestJSON.test_list_networks_hosted_by_one_dhcpsame 

neutron_tempest_plugin.api.admin.test_external_network_extension.ExternalNetworksRBACTestJSON.test_delete_policies_while_tenant_attached_to_netsame

neutron_tempest_plugin.api.admin.test_external_network_extension.ExternalNetworksRBACTestJSON.test_external_update_policy_from_wildcard_to_specific_tenantsame

neutron_tempest_plugin.api.admin.test_external_network_extension.ExternalNetworksRBACTestJSON.test_policy_allows_tenant_to_allocate_floatingipsame

neutron_tempest_plugin.api.admin.test_external_network_extension.ExternalNetworksRBACTestJSON.test_policy_allows_tenant_to_attach_ext_gwsame

neutron_tempest_plugin.api.test_dhcp_ipv6.NetworksTestDHCPv6.test_dhcp_stateful_fixedips_outrangesame

neutron_tempest_plugin.api.test_timestamp.TestTimeStampWithL3.test_create_floatingip_with_timestampsame

neutron_tempest_plugin.api.test_timestamp.TestTimeStampWithL3.test_create_router_with_timestampsame

neutron_tempest_plugin.api.test_timestamp.TestTimeStampWithL3.test_show_floatingip_attribute_with_timestampsame

neutron_tempest_plugin.api.test_timestamp.TestTimeStampWithL3.test_show_router_attribute_with_timestampsame

neutron_tempest_plugin.api.test_timestamp.TestTimeStampWithL3.test_update_floatingip_with_timestampsame

neutron_tempest_plugin.api.test_timestamp.TestTimeStampWithL3.test_update_router_with_timestampsame

QOS test casesUpstream: QOS support in DPDK

neutron_tempest_plugin.api.test_security_groups.RbacSharedSecurityGroupTest.test_regular_client_shares_to_another_regular_clientTBD
airship-opnfv-functest-smoke-latest-tempest_cinder-run


cinder_tempest_plugin.api.volume.test_volume_revert.VolumeRevertTests.test_volume_revert_to_snapshotAIRSHIP-96: upstream bug related to kernel 4.15
airship-opnfv-functest-smoke-latest-tempest_keystone-run

airship-opnfv-functest-smoke-latest-refstack_platform-run
skipped
airship-opnfv-functest-
smoke-latest-tempest_full-runairship-opnfv-functest-smoke-latest-tempest_heat-run
smoke-latest-tempest_full-run


tempest.api.compute.admin.test_instance_usage_audit_log_negative.InstanceUsageAuditLogNegativeTestJSON.test_instance_usage_audit_logs_with_nonadmin_user

AIRSHIP-104: Airship rbac policy inconsistent with tempest default


empest.api.compute.admin.test_live_migration.LiveAutoBlockMigrationV225Test.test_live_block_migration

1289064: upstream: live migration is broken in numa. Needs Train


tempest.api.compute.admin.test_live_migration.LiveAutoBlockMigrationV225Test.test_live_block_migration_paused

same


tempest.api.compute.admin.test_live_migration.LiveMigrationTest.test_live_block_migration

same


tempest.api.compute.admin.test_live_migration.LiveMigrationTest.test_live_block_migration_paused

same


tempest.api.compute.admin.test_migrations.MigrationsAdminTest.test_list_migrations_in_flavor_resize_situation

TBD


tempest.api.compute.admin.test_migrations.MigrationsAdminTest.test_resize_server_revert_deleted_flavor

TBD


tempest.api.compute.admin.test_simple_tenant_usage.TenantUsagesTestJSON.test_get_usage_tenant_with_non_admin_user

AIRSHIP-105: Airship rbac policy inconsistent with tempest default


tempest.api.compute.servers.test_attach_interfaces.AttachInterfacesTestJSON.test_create_list_show_delete_interfaces_by_network_port

fail 36.011


tempest.api.compute.servers.test_attach_interfaces.AttachInterfacesTestJSON.test_reassign_port_between_servers

fail 89.588


tempest.api.compute.servers.test_create_server_multi_nic.ServersTestMultiNic.test_verify_duplicate_network_nics

fail 126.883


tempest.api.compute.servers.test_create_server_multi_nic.ServersTestMultiNic.test_verify_multiple_nics_order

fail 125.284





tempest.api.compute.servers.test_delete_server.DeleteServersTestJSON.test_delete_server_while_in_verify_resize_state

TBD


tempest.api.compute.servers.test_device_tagging.TaggedAttachmentsTest.test_tagged_attachment

TBD


tempest.api.compute.servers.test_disk_config.ServerDiskConfigTestJSON.test_resize_server_from_auto_to_manual

TBD


tempest.api.compute.servers.test_disk_config.ServerDiskConfigTestJSON.test_resize_server_from_manual_to_auto

TBD


tempest.api.compute.servers.test_server_actions.ServerActionsTestJSON.test_get_console_output

Upstream bug: AIRSHIP-95. Nova instance console log api returns emptry


tempest.api.compute.servers.test_server_actions.ServerActionsTestJSON.test_get_console_output_server_id_in_shutoff_status

same


tempest.api.compute.servers.test_server_actions.ServerActionsTestJSON.test_get_console_output_with_unlimited_size

same


tempest.api.compute.servers.test_server_actions.ServerActionsTestJSON.test_resize_server_confirm

TBD


tempest.api.compute.servers.test_server_actions.ServerActionsTestJSON.test_resize_server_confirm_from_stopped

TBD


tempest.api.compute.servers.test_server_actions.ServerActionsTestJSON.test_resize_server_revert

TBD


tempest.api.compute.servers.test_server_actions.ServerActionsTestJSON.test_resize_server_revert_with_volume_attached

TBD


tempest.api.compute.servers.test_server_actions.ServerActionsTestJSON.test_resize_volume_backed_server_confirm

TBD


tempest.api.identity.admin.v3.test_application_credentials.ApplicationCredentialsV3AdminTest.test_create_application_credential_with_rolesAIRSHIP-106: Airship rbac policy inconsistent with tempest default

tempest.api.identity.admin.v3.test_groups.GroupsV3TestJSON.test_list_groupsAIRSHIP-107: possible rbac?

tempest.api.identity.admin.v3.test_list_users.UsersV3TestJSON.test_list_userszAIRSHIP-108: possible rbac

tempest.api.identity.admin.v3.test_list_users.UsersV3TestJSON.test_list_users_with_namesame

tempest.api.identity.admin.v3.test_projects_negative.ProjectsNegativeStaticTestJSON.test_list_projects_by_unauthorized_user

AIRSHIP-109: airship rbac policy is different than tempest default


tempest.api.identity.v3.test_application_credentials.ApplicationCredentialsV3Test.test_create_application_credentialAIRSHIP-110: airship rbac policy is different than tempest default

tempest.api.identity.v3.test_application_credentials.ApplicationCredentialsV3Test.test_create_application_credential_expiressame

tempest.api.identity.v3.test_application_credentials.ApplicationCredentialsV3Test.test_list_application_credentialssame

tempest.api.identity.v3.test_application_credentials.ApplicationCredentialsV3Test.test_query_application_credentialssame

tempest.api.identity.v3.test_users.IdentityV3UsersTest.test_password_history_check_self_service_api

TBD: possible rbac


tempest.api.identity.v3.test_users.IdentityV3UsersTest.test_user_account_lockout

TBD: possible rbac


tempest.api.identity.v3.test_users.IdentityV3UsersTest.test_user_update_own_password

TBD: possible rbac


tempest.api.image.v2.test_images.BasicOperationsImagesTest.test_deactivate_reactivate_imageTBD: possible rbac

tempest.api.network.admin.test_dhcp_agent_scheduler.DHCPAgentSchedulersTestJSON.test_list_networks_hosted_by_one_dhcpFUNCTEST-1023: L3 test cases need to be skipped based on Neutron extension list or blacklisted

tempest.api.network.admin.test_external_network_extension.ExternalNetworksTestJSON.test_list_external_networkssame

tempest.api.network.admin.test_external_network_extension.ExternalNetworksTestJSON.test_show_external_networks_attributesame

tempest.api.network.test_dhcp_ipv6.NetworksTestDHCPv6.test_dhcp_statefulsame

tempest.api.network.test_dhcp_ipv6.NetworksTestDHCPv6.test_dhcp_stateful_fixedipssame

tempest.api.network.test_dhcp_ipv6.NetworksTestDHCPv6.test_dhcp_stateful_fixedips_duplicatesame

tempest.api.network.test_dhcp_ipv6.NetworksTestDHCPv6.test_dhcp_stateful_fixedips_outrangesame

tempest.api.network.test_dhcp_ipv6.NetworksTestDHCPv6.test_dhcp_stateful_routersame

tempest.api.network.test_dhcp_ipv6.NetworksTestDHCPv6.test_dhcpv6_64_subnetssame

tempest.api.network.test_dhcp_ipv6.NetworksTestDHCPv6.test_dhcpv6_invalid_optionssame

tempest.api.network.test_dhcp_ipv6.NetworksTestDHCPv6.test_dhcpv6_stateless_eui64same

tempest.api.network.test_dhcp_ipv6.NetworksTestDHCPv6.test_dhcpv6_stateless_no_rasame

tempest.api.network.test_dhcp_ipv6.NetworksTestDHCPv6.test_dhcpv6_stateless_no_ra_no_dhcpsame

tempest.api.network.test_dhcp_ipv6.NetworksTestDHCPv6.test_dhcpv6_two_subnetssame

tempest.api.network.test_ports.PortsIpV6TestJSON.test_port_list_filter_by_router_idsame

tempest.api.network.test_ports.PortsTestJSON.test_port_list_filter_by_router_idsame

tempest.api.volume.admin.test_volume_manage.VolumeManageAdminTest.test_unmanage_manage_volume

TBD


tempest.api.volume.test_volumes_negative.VolumesNegativeTest.test_create_volume_from_deactivated_image

AIRSHIP-112: possible rbac?










airship-opnfv-functest-smoke-latest-tempest_heat-run


heat_tempest_plugin.tests.functional.test_nova_server_networks.CreateServerTest.test_create_server_with_portFUNCTEST_1022: Requires neutron l3 extension not supported in Airship DPDK. Should be dynamically skipped based on neutron extension list.

heat_tempest_plugin.tests.functional.test_nova_server_networks.CreateServerTest.test_create_server_with_subnet_fixed_ip_sec_group

heat_tempest_plugin.tests.functional.test_nova_server_networks.CreateServerTest.test_create_update_server_with_subnet

heat_tempest_plugin.tests.functional.test_nova_server_networks.UpdateServerNetworksTest.test_create_update_server_add_network

heat_tempest_plugin.tests.functional.test_nova_server_networks.UpdateServerNetworksTest.test_create_update_server_add_same_fixed_ip

heat_tempest_plugin.tests.functional.test_nova_server_networks.UpdateServerNetworksTest.test_create_update_server_add_subnet

heat_tempest_plugin.tests.functional.test_nova_server_networks.UpdateServerNetworksTest.test_create_update_server_multi_networks_swaps

heat_tempest_plugin.tests.functional.test_nova_server_networks.
CreateServerTest
UpdateServerNetworksTest.test_create_update_server_swap_
with
network_port
Requires neutron l3 extension not supported in Airship DPDK


heat_tempest_plugin.tests.functional.test_nova_server_networks.
CreateServerTest
UpdateServerNetworksTest.test_create_update_server_
with_subnet_fixed_ip_sec_group
swap_network_subnet

heat_tempest_plugin.tests.functional.test_nova_server_networks.
CreateServerTest
UpdateServerNetworksTest.test_create_update_server_
with
swap_subnet_network

heat_tempest_plugin.tests.
functional
scenario.test
_nova
_server_
networks
signal.
UpdateServerNetworksTest
ServerSignalIntegrationTest.test_
create
server_
update
signal_
server
userdata_
add
format_
network
raw

heat_tempest_plugin.tests.
functional
scenario.test
_nova
_server_
networks
signal.
UpdateServerNetworksTest
ServerSignalIntegrationTest.test_
create
server_
update
signal_
server
userdata_
add
format_
same
software_
fixed_ip
config

heat_tempest_plugin.tests.functional.test_
nova
os_
server
wait_
networks
condition.
UpdateServerNetworksTest
OSWaitCondition.test_create_stack_
update
with_
server
multi_
add
signal_
subnet
waitconditionTBD

heat_tempest_plugin.tests.
functional
scenario.test_
nova
base_
server_networks
resources.
UpdateServerNetworksTest
BasicResourcesTest.test_
create
base_
update_server_multi_networks_swaps
resources_integrationTBD

heat_tempest_plugin.tests.
functional
scenario.test_
nova
remote_
server
deeply_
networks
nested.
UpdateServerNetworksTest
RemoteDeeplyNestedStackTest.test_
create_update_server_swap_network_portheat_tempest_plugin.tests.functional.test_nova_server_networks.UpdateServerNetworksTest.test_create_update_server_swap_network_subnetheat_tempest_plugin.tests.functional.test_nova_server_networks.UpdateServerNetworksTest.test_create_update_server_swap_subnet_networkheat_tempest_plugin.tests.scenario.test_server_signal.ServerSignalIntegrationTest.test_server_signal_userdata_format_rawheat_tempest_plugin.tests.scenario.test_server_signal.ServerSignalIntegrationTest.test_server_signal_userdata_format_software_configheat
remote_nestedTBD
airship-opnfv-functest-smoke-latest-tempest_scenario-run


tempest.scenario.test_server_basic_ops.TestServerBasicOps.test_server_basic_opsUpstream bug: AIRSHIP-103. Nova metaservice url not reachable from guest vm
airship-opnfv-functest-smoke-latest-tempest_telemetry-run
skipped
airship-opnfv-functest-smoke-latest-tempest_slow-run


tempest.scenario.test_volume_backup_restore.TestVolumeBackupRestore.test_volume_backup_restoreFUNCTEST-1019: teste volume backup depends on floating ip

tempest.api.compute.servers.test_device_tagging.TaggedBootDevicesTest.test_tagged_boot_devicesFUNCTEST-1020: test device tagging still uses tenant network
airship-opnfv-functest-smoke-latest-rally_sanity-run


NovaServers.boot_and_live_migrate_serverNo support from OpenStack Helm. Recommend to blacklist.
airship-opnfv-functest-smoke-latest-patrole-run


patrole_tempest_plugin.tests.
functional
api.network.test_
os
floating_
wait
ips_
condition
rbac.
OSWaitCondition
FloatingIpsRbacTest.test_create_
stack_with_multi_signal_waitconditionTBDheat_tempest_plugin.tests.scenario.test_base_resources.BasicResourcesTest.test_base_resources_integrationTBDheat
floating_ipFUNCTEST-1021: Requires neutron l3 extension not supported in Airship DPDK. Should be dynamically skipped based on neutron extension list.

patrole_tempest_plugin.tests.api.
scenario
network.test_
remote
floating_
deeply
ips_
nested
rbac.
RemoteDeeplyNestedStackTest
FloatingIpsRbacTest.test_create_floating_
remote_nestedTBDairship-opnfv-functest-smoke-latest-tempest_scenario-runtempest.scenario.test_server_basic_ops.TestServerBasicOps.test_server_basic_opsUpstream bug: AIRSHIP-103. Nova metaservice url not reachable from guest vmairship-opnfv-functest-smoke-latest-tempest_telemetry-runskippedairship-opnfv-functest-smoke-latest-tempest_slow-runtempest.scenario.test_volume_backup_restore.TestVolumeBackupRestore.test_volume_backup_restoreFUNCTEST-1019: teste volume backup depends on floating iptempest.api.compute.servers.test_device_tagging.TaggedBootDevicesTest.test_tagged_boot_devicesFUNCTEST-1020: test device tagging still uses tenant networkairship-opnfv-functest-smoke-latest-rally_sanity-runNovaServers.boot_and_live_migrate_serverNo support from OpenStack Helm. Recommend to blacklist.airship-opnfv-functest-smoke-latest-patrole-run
ip_floatingip_address

patrole_tempest_plugin.tests.api.network.test_floating_ips_rbac.FloatingIpsRbacTest.test_delete_floating_ip

patrole_tempest_plugin.tests.api.network.test_floating_ips_rbac.FloatingIpsRbacTest.test_list_floating_ips

patrole_tempest_plugin.tests.api.network.test_floating_ips_rbac.FloatingIpsRbacTest.test_show_floating_ip

patrole_tempest_plugin.tests.api.network.test_floating_ips_rbac.FloatingIpsRbacTest.test_update_floating_ip

patrole_tempest_plugin.tests.api.network.test_networks_rbac.NetworksRbacTest.test_create_network_is_default
airship-opnfv-functest-smoke-latest-refstack_compute-run
skipped
airship-opnfv-functest-smoke-latest-refstack_object-run

airship-opnfv-functest-smoke-latest-networking-sfc-run
skipped
airship-opnfv-functest-smoke-latest-networking-bgpvpn-run
skipped
airship-opnfv-functest-smoke-latest-tempest_barbican-run
skipped
airship-opnfv-functest-smoke-latest-tempest_octavia-run
skipped
Benchmark

airship-opnfv-functest-benchmarking-latest-rally_full-run


NeutronNetworks.create_and_show_ports

needs blacklist. It needs router extension.


NovaServers.resize_serverSLA. Retest on pod 15. pod17 too small.

GlanceImages.create_and_deactivate_imageSLA. Retest on pod 15. pod17 too small.
airship-opnfv-functest-benchmarking-latest-rally_jobs-run
skipped
airship-opnfv-functest-benchmarking-latest-vmtp-run
skipped
airship-opnfv-functest-benchmarking-latest-shaker-run
skipped
VNF

airship-opnfv-functest-vnf-latest-cloudify-run
FUNCTEST-1017: cloudify vpn deploy uses floating ip. Shoudl change to use "public" network or skip.
airship-opnfv-functest-vnf-latest-cloudify_ims-run
skipped
airship-opnfv-functest-vnf-latest-heat_ims-run
skipped
airship-opnfv-functest-vnf-latest-vyos_vrouter-run
skipped
airship-opnfv-functest-vnf-latest-juju_epc-run
skipped