Commit c0cf70a6 authored by Jacob Keller's avatar Jacob Keller Committed by Jeff Kirsher

i40e: don't add more vectors to num_lan_msix than number of CPUs

This is a solution to avoid adding too many queues to num_lan_msix.
A recent refactor of queue pairs accidentally added all remaining
vectors to the num_lan_msix which can have adverse performance issues,
due to enabling more queues than the number of CPU cores.

This patch removes the old calculation, and replaces it with a simple
algorithm.

1) add queue pairs up to num_online_cpus(), but capped at half of total
   vectors
2) then add alternative features such as flow directory and similar
3) finally, add the remaining vectors back to queue pairs, but capped
   such that the total number of queue pairs does not exceed
   num_online_cpus().

Change-ID: I668abf67d5011a1248866daba8885f4ff00cb8d9
Signed-off-by: default avatarJacob Keller <jacob.e.keller@intel.com>
Signed-off-by: default avatarHarshitha Ramamurthy <harshitha.ramamurthy@intel.com>
Signed-off-by: default avatarCarolyn Wyborny <carolyn.wyborny@intel.com>
Tested-by: default avatarAndrew Bowers <andrewx.bowers@intel.com>
Signed-off-by: default avatarJeff Kirsher <jeffrey.t.kirsher@intel.com>
parent 0ef2d5af
...@@ -7819,6 +7819,7 @@ static int i40e_reserve_msix_vectors(struct i40e_pf *pf, int vectors) ...@@ -7819,6 +7819,7 @@ static int i40e_reserve_msix_vectors(struct i40e_pf *pf, int vectors)
static int i40e_init_msix(struct i40e_pf *pf) static int i40e_init_msix(struct i40e_pf *pf)
{ {
struct i40e_hw *hw = &pf->hw; struct i40e_hw *hw = &pf->hw;
int cpus, extra_vectors;
int vectors_left; int vectors_left;
int v_budget, i; int v_budget, i;
int v_actual; int v_actual;
...@@ -7854,10 +7855,16 @@ static int i40e_init_msix(struct i40e_pf *pf) ...@@ -7854,10 +7855,16 @@ static int i40e_init_msix(struct i40e_pf *pf)
vectors_left--; vectors_left--;
} }
/* reserve vectors for the main PF traffic queues */ /* reserve some vectors for the main PF traffic queues. Initially we
pf->num_lan_msix = min_t(int, num_online_cpus(), vectors_left); * only reserve at most 50% of the available vectors, in the case that
* the number of online CPUs is large. This ensures that we can enable
* extra features as well. Once we've enabled the other features, we
* will use any remaining vectors to reach as close as we can to the
* number of online CPUs.
*/
cpus = num_online_cpus();
pf->num_lan_msix = min_t(int, cpus, vectors_left / 2);
vectors_left -= pf->num_lan_msix; vectors_left -= pf->num_lan_msix;
v_budget += pf->num_lan_msix;
/* reserve one vector for sideband flow director */ /* reserve one vector for sideband flow director */
if (pf->flags & I40E_FLAG_FD_SB_ENABLED) { if (pf->flags & I40E_FLAG_FD_SB_ENABLED) {
...@@ -7920,6 +7927,23 @@ static int i40e_init_msix(struct i40e_pf *pf) ...@@ -7920,6 +7927,23 @@ static int i40e_init_msix(struct i40e_pf *pf)
} }
} }
/* On systems with a large number of SMP cores, we previously limited
* the number of vectors for num_lan_msix to be at most 50% of the
* available vectors, to allow for other features. Now, we add back
* the remaining vectors. However, we ensure that the total
* num_lan_msix will not exceed num_online_cpus(). To do this, we
* calculate the number of vectors we can add without going over the
* cap of CPUs. For systems with a small number of CPUs this will be
* zero.
*/
extra_vectors = min_t(int, cpus - pf->num_lan_msix, vectors_left);
pf->num_lan_msix += extra_vectors;
vectors_left -= extra_vectors;
WARN(vectors_left < 0,
"Calculation of remaining vectors underflowed. This is an accounting bug when determining total MSI-X vectors.\n");
v_budget += pf->num_lan_msix;
pf->msix_entries = kcalloc(v_budget, sizeof(struct msix_entry), pf->msix_entries = kcalloc(v_budget, sizeof(struct msix_entry),
GFP_KERNEL); GFP_KERNEL);
if (!pf->msix_entries) if (!pf->msix_entries)
......
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment