1. 14 Aug, 2018 5 commits
  2. 13 Aug, 2018 5 commits
    • Jason Gunthorpe's avatar
      IB/uverbs: Do not check for device disassociation during ioctl · 4ce719f8
      Jason Gunthorpe authored
      Now that the ioctl path and uobjects are converted to use uverbs_api, it
      is now safe to remove the disassociation protection from the common ioctl
      code.
      
      This completes the work to make destroy functions continue to work even
      after device disassociation.
      Signed-off-by: default avatarJason Gunthorpe <jgg@mellanox.com>
      4ce719f8
    • Jason Gunthorpe's avatar
      IB/uverbs: Remove struct uverbs_root_spec and all supporting code · 51d0a2b4
      Jason Gunthorpe authored
      Everything now uses the uverbs_uapi data structure.
      Signed-off-by: default avatarJason Gunthorpe <jgg@mellanox.com>
      51d0a2b4
    • Jason Gunthorpe's avatar
      IB/uverbs: Use uverbs_api to unmarshal ioctl commands · 3a863577
      Jason Gunthorpe authored
      Convert the ioctl method syscall path to use the uverbs_api data
      structures. The new uapi structure includes all the same information, just
      in a different and more optimal way.
      
       - Use attr_bkey instead of 2 level radix trees for everything related to
         attributes. This includes the attribute storage, presence, and
         detection of missing mandatory attributes.
       - Avoid iterating over all attribute storage at finish, instead use
         find_first_bit with the attr_bkey to locate only those attrs that need
         cleanup.
       - Organize things to always run, and always rely on, cleanup. This
         avoids a bunch of tricky error unwind cases.
       - Locate the method using the radix tree, and locate the attributes
         using a very efficient incremental radix tree lookup
       - Use the precomputed destroy_bkey to handle uobject destruction
       - Use the precomputed allocation sizes and precomputed 'need_stack'
         to avoid maths in the fast path. This is optimal if userspace
         does not pass (many) unsupported attributes.
      
      Overall this results in much better codegen for the attribute accessors,
      everything is now stored in bitmaps or linear arrays indexed by attr_bkey.
      The compiler can compute attr_bkey values at compile time for all method
      attributes, meaning things like uverbs_attr_is_valid() now compile into
      single instruction bit tests.
      Signed-off-by: default avatarJason Gunthorpe <jgg@mellanox.com>
      3a863577
    • Jason Gunthorpe's avatar
      IB/uverbs: Use uverbs_alloc for allocations · b61815e2
      Jason Gunthorpe authored
      Several handlers need temporary allocations for the life of the method,
      switch them to use the uverbs_alloc allocator.
      Signed-off-by: default avatarJason Gunthorpe <jgg@mellanox.com>
      Reviewed-by: default avatarLeon Romanovsky <leonro@mellanox.com>
      b61815e2
    • Jason Gunthorpe's avatar
      IB/uverbs: Add a simple allocator to uverbs_attr_bundle · 461bb2ee
      Jason Gunthorpe authored
      This is similar in spirit to devm, it keeps track of any allocations
      linked to this method call and ensures they are all freed when the method
      exits. Further, if there is space in the internal/onstack buffer then the
      allocator will hand out that memory and avoid an expensive call to
      kalloc/kfree in the syscall path.
      Signed-off-by: default avatarJason Gunthorpe <jgg@mellanox.com>
      Reviewed-by: default avatarLeon Romanovsky <leonro@mellanox.com>
      461bb2ee
  3. 10 Aug, 2018 5 commits
    • Jason Gunthorpe's avatar
      IB/uverbs: Remove the ib_uverbs_attr pointer from each attr · 6a1f444f
      Jason Gunthorpe authored
      Memory in the bundle is valuable, do not waste it holding an 8 byte
      pointer for the rare case of writing to a PTR_OUT. We can compute the
      pointer by storing a small 1 byte array offset and the base address of the
      uattr memory in the bundle private memory.
      
      This also means we can access the kernel's copy of the ib_uverbs_attr, so
      drop the copy of flags as well.
      
      Since the uattr base should be private bundle information this also
      de-inlines the already too big uverbs_copy_to inline and moves
      create_udata into uverbs_ioctl.c so they can see the private struct
      definition.
      Signed-off-by: default avatarJason Gunthorpe <jgg@mellanox.com>
      Reviewed-by: default avatarLeon Romanovsky <leonro@mellanox.com>
      6a1f444f
    • Jason Gunthorpe's avatar
      IB/uverbs: Provide implementation private memory for the uverbs_attr_bundle · 4b3dd2bb
      Jason Gunthorpe authored
      This already existed as the anonymous 'ctx' structure, but this was not
      really a useful form. Hoist this struct into bundle_priv and rework the
      internal things to use it instead.
      
      Move a bunch of the processing internal state into the priv and reduce the
      excessive use of function arguments.
      Signed-off-by: default avatarJason Gunthorpe <jgg@mellanox.com>
      Reviewed-by: default avatarLeon Romanovsky <leonro@mellanox.com>
      4b3dd2bb
    • Jason Gunthorpe's avatar
      IB/uverbs: Use uverbs_api to manage the object type inside the uobject · 6b0d08f4
      Jason Gunthorpe authored
      Currently the struct uverbs_obj_type stored in the ib_uobject is part of
      the .rodata segment of the module that defines the object. This is a
      problem if drivers define new uapi objects as we will be left with a
      dangling pointer after device disassociation.
      
      Switch the uverbs_obj_type for struct uverbs_api_object, which is
      allocated memory that is part of the uverbs_api and is guaranteed to
      always exist. Further this moves the 'type_class' into this memory which
      means access to the IDR/FD function pointers is also guaranteed. Drivers
      cannot define new types.
      
      This makes it safe to continue to use all uobjects, including driver
      defined ones, after disassociation.
      Signed-off-by: default avatarJason Gunthorpe <jgg@mellanox.com>
      6b0d08f4
    • Jason Gunthorpe's avatar
      IB/uverbs: Build the specs into a radix tree at runtime · 9ed3e5f4
      Jason Gunthorpe authored
      This radix tree datastructure is intended to replace the 'hash' structure
      used today for parsing ioctl methods during system calls. This first
      commit introduces the structure and builds it from the existing .rodata
      descriptions.
      
      The so-called hash arrangement is actually a 5 level open coded radix tree.
      This new version uses a 3 level radix tree built using the radix tree
      library.
      
      Overall this is much less code and much easier to build as the radix tree
      API allows for dynamic modification during the building. There is a small
      memory penalty to pay for this, but since the radix tree is allocated on
      a per device basis, a few kb of RAM seems immaterial considering the
      gained simplicity.
      
      The radix tree is similar to the existing tree, but also has a 'attr_bkey'
      concept, which is a small value'd index for each method attribute. This is
      used to simplify and improve performance of everything in the next
      patches.
      Signed-off-by: default avatarJason Gunthorpe <jgg@mellanox.com>
      Reviewed-by: default avatarLeon Romanovsky <leonro@mellanox.com>
      Reviewed-by: default avatarMichael J. Ruhl <michael.j.ruhl@intel.com>
      9ed3e5f4
    • Jason Gunthorpe's avatar
      IB/uverbs: Have the core code create the uverbs_root_spec · 7d96c9b1
      Jason Gunthorpe authored
      There is no reason for drivers to do this, the core code should take of
      everything. The drivers will provide their information from rodata to
      describe their modifications to the core's base uapi specification.
      
      The core uses this to build up the runtime uapi for each device.
      Signed-off-by: default avatarJason Gunthorpe <jgg@mellanox.com>
      Reviewed-by: default avatarMichael J. Ruhl <michael.j.ruhl@intel.com>
      Reviewed-by: default avatarLeon Romanovsky <leonro@mellanox.com>
      7d96c9b1
  4. 09 Aug, 2018 1 commit
  5. 08 Aug, 2018 5 commits
  6. 07 Aug, 2018 1 commit
  7. 03 Aug, 2018 13 commits
  8. 01 Aug, 2018 5 commits
    • Denis Drozdov's avatar
      IB/IPoIB: Set ah valid flag in multicast send flow · 75da9606
      Denis Drozdov authored
      The change of ipoib_ah data structure with adding "valid" flag and
      checks of ah->valid in ipoib_start_xmit affected multicast packet flow.
      
      Since the multicast flow doesn't invoke path_rec_start, "ah->valid" flag
      remains unset, so that ipoib_start_xmit end up with neigh_refresh_path
      instead of sending the packet using neigh.
      
      "ah->valid" has to be set in multicast send flow. As a result IPoIB
      starts sending packets via neigh immediately and eliminates 60sec delay
      of neigh keep alive interval.
      
      The typical example of this issue are two sequential arpings:
      
      arping 11.134.208.9 -> got response (mcast_send)
      arping 11.134.208.9 -> no response  (ah->valid = 0)
      
      Fixes: fa9391db ("RDMA/ipoib: Update paths on CLIENT_REREG/SM_CHANGE events")
      Signed-off-by: default avatarDenis Drozdov <denisd@mellanox.com>
      Reviewed-by: default avatarErez Shitrit <erezsh@mellanox.com>
      Reviewed-by: default avatarFeras Daoud <ferasda@mellanox.com>
      Signed-off-by: default avatarLeon Romanovsky <leonro@mellanox.com>
      Signed-off-by: default avatarJason Gunthorpe <jgg@mellanox.com>
      75da9606
    • Jason Gunthorpe's avatar
      IB/uverbs: Allow all DESTROY commands to succeed after disassociate · 0f50d88a
      Jason Gunthorpe authored
      The disassociate function was broken by design because it failed all
      commands. This prevents userspace from calling destroy on a uobject after
      it has detected a device fatal error and thus reclaiming the resources in
      userspace is prevented.
      
      This fix is now straightforward, when anything destroys a uobject that is
      not the user the object remains on the IDR with a NULL context and object
      pointer. All lookup locking modes other than DESTROY will fail. When the
      user ultimately calls the destroy function it is simply dropped from the
      IDR while any related information is returned.
      Signed-off-by: default avatarJason Gunthorpe <jgg@mellanox.com>
      0f50d88a
    • Jason Gunthorpe's avatar
      IB/uverbs: Do not block disassociate during write() · a9b66d64
      Jason Gunthorpe authored
      Now that all the callbacks are safe to run concurrently with
      disassociation this test can be eliminated. The ufile core infrastructure
      becomes entirely self contained and is not sensitive to disassociation.
      Signed-off-by: default avatarJason Gunthorpe <jgg@mellanox.com>
      a9b66d64
    • Jason Gunthorpe's avatar
      IB/uverbs: Do not pass struct ib_device to the ioctl methods · e83f0ecd
      Jason Gunthorpe authored
      This does the same as the patch before, except for ioctl. The rules are
      the same, but for the ioctl methods the core code handles setting up the
      uobject.
      
      - Retrieve the ib_dev from the uobject->context->device. This is
        safe under ioctl as the core has already done rdma_alloc_begin_uobject
        and so CREATE calls are entirely protected by the rwsem.
      - Retrieve the ib_dev from uobject->object
      - Call ib_uverbs_get_ucontext()
      Signed-off-by: default avatarJason Gunthorpe <jgg@mellanox.com>
      e83f0ecd
    • Jason Gunthorpe's avatar
      IB/uverbs: Do not pass struct ib_device to the write based methods · bbd51e88
      Jason Gunthorpe authored
      This is a step to get rid of the global check for disassociation. In this
      model, the ib_dev is not proven to be valid by the core code and cannot be
      provided to the method. Instead, every method decides if it is able to
      run after disassociation and obtains the ib_dev using one of three
      different approaches:
      
      - Call srcu_dereference on the udevice's ib_dev. As before, this means
        the method cannot be called after disassociation begins.
        (eg alloc ucontext)
      - Retrieve the ib_dev from the ucontext, via ib_uverbs_get_ucontext()
      - Retrieve the ib_dev from the uobject->object after checking
        under SRCU if disassociation has started (eg uobj_get)
      
      Largely, the code is all ready for this, the main work is to provide a
      ib_dev after calling uobj_alloc(). The few other places simply use
      ib_uverbs_get_ucontext() to get the ib_dev.
      
      This flexibility will let the next patches allow destroy to operate
      after disassociation.
      Signed-off-by: default avatarJason Gunthorpe <jgg@mellanox.com>
      bbd51e88