1. 11 Nov, 2024 6 commits
    • Kirill Smelkov's avatar
      amari.drb: Start tracking current bitrate in per-cell UE transmission state · 91967123
      Kirill Smelkov authored
      We will need to know current cell DL/UL bitrate for multicell BitSync in
      the next patches.
      91967123
    • Kirill Smelkov's avatar
      amari.drb: Split _Utx into global and per-cell parts · 26a82c6e
      Kirill Smelkov authored
      Continue preparatory steps to support multicell configurations and for
      that split the class that tracks UE transmission state into global part
      (_Utx) and per-cell parts (_UCtx).
      
      Everywhere else in the code we still assert that the number of cells an UE
      attached to is 1, so no support for multicell yet - only preparatory
      non-functional changes currently.
      26a82c6e
    • Kirill Smelkov's avatar
      amari.drb: Move tracking of #tx and #retx into _Utx · bd57acbb
      Kirill Smelkov authored
      Similarly to previous patch this is preparatory non-functional change to
      support multicell configurations.
      
      Previously the number of transmitted transport blocks was passed around
      as separate argument but with multiple cells each cell will have its own
      information about how many TB were transmitted/received and we will need
      to maintain those .tx and .retx in per-cell data structure.
      
      Start preparing to that by moving .tx and .retx to be tracked in UE
      transmission state instead of being passed around separately.
      
      No support for multicell yet - only preparatory non-functional changes currently.
      bd57acbb
    • Kirill Smelkov's avatar
      amari.drb: Move handling of xl_use_avg and rank from _QCI_Flow to higher level at _UE · beeb9dea
      Kirill Smelkov authored
      Since 2a016d48 (Draft support for E-UTRAN IP Throughput KPI) there was a
      hardcoded limitation that x.drb_stats generation works with 1-cell
      configurations only. However we do use multicell eNB configurations and
      on such configurations `xamari xlog x.drb_stats` was failing on eNB
      side with
      
          raise RuntimeError(("ue #%s belongs to %d cells;  "+
              "but only single-cell configurations are supported") % (ue_id, len(ju(['cells']))))
      
      Start fixing that.
      
      As the first preparatory step to support multiple cells with x.drb_stats
      reorganize amari/drb.py code a bit: _QCI_Flow works at whole QCI
      transmission level which can aggregate and cover multiple cells, but
      xl_use_avg and rank are per cell things and they can be different for
      different cells.
      
      -> Start preparing to handle them in per-cell way by adjusting the code
         to take those values into account at higher level in computation stack
         where we still have cell context.
      
      No support for multicell yet - only preparatory non-functional changes
      currently.
      beeb9dea
    • Kirill Smelkov's avatar
      amari.kpi: Fix LogMeasure to handle multiple x.drb_stats messages in one period · 1b9d4e1a
      Kirill Smelkov authored
      When computing Measurements amari.kpi derives period from stats messages
      - for example 60s if `xamari xlog` was running with stats/60s. But
      inside that period there could be multiple x.drb_stats messages - for
      example if xlog was running with `stats/60s x.drb_stats/10s`.
      However the code was handling only the last such x.drb_stats instead of
      accumulating counters from there during the period.
      
      As the result, if. e.g. during the period there were 90B/9s and 2B/0.2s
      x.drb_stats events, only the latter was accounted giving wrong data for
      transmitted amount and transmission time.
      
      -> Fix it.
      1b9d4e1a
    • Kirill Smelkov's avatar
      amari.kpi: Teach LogMeasure to handle stats messages with multiple cells · 51c01b14
      Kirill Smelkov authored
      Starting from 71087f67 (amari.kpi: New package with driver for Amarisoft
      LTE stack to retrieve KPI-related measurements from logs) a limitation
      was hardcoded that only 1-cell configurations are supported. However we
      do use multicell eNB configurations and computing E-RAB Accessibility
      KPI was failing on them as e.g.:
      
          xlte.amari.kpi.LogError: t1731059787.321: stats describes 2 cells;  but only single-cell configurations are supported
      
      -> Remove that limitation and handle stats covering multiple cells.
      51c01b14
  2. 27 May, 2024 2 commits
    • Kirill Smelkov's avatar
      amari.xlog: Add support for password-based authentication · c5e92b6a
      Kirill Smelkov authored
      Add new --password option and wire it to go to amari.connect(password=...)
      
      Support for password-based authentication in amari.connect was just
      added in the previous patch.
      
      We need to extend filtering of logged fields on service attach a bit
      since now the first service message can be both 'ready' and
      'authenticate', and besides e.g. 'message' we don't want to log what was
      the 'challenge'.
      
      /reported-and-tested-by @lu.xu
      /reviewed-on kirr/xlte!6
      c5e92b6a
    • Kirill Smelkov's avatar
      amari: Conn: add support for password-based authentication · 5eb80f9a
      Kirill Smelkov authored
      Sometimes Amarisoft services are setup to require custom
      challenge/response authentication upon connecting to their WebSocket.
      In 61ad9032 (amari: Add functionality to interoperate with an Amarisoft
      LTE service via WebSocket) I've put TODO for that but now we start to
      need it.
      
      -> Implement corresponding support for password-based authentication.
      
      In the next patch we will teach XLog to use it.
      
      See https://tech-academy.amarisoft.com/lteenb.doc#Startup for details of
      the authentication handshake protocol.
      
      /reported-and-tested-by @lu.xu
      /reviewed-on kirr/xlte!6
      5eb80f9a
  3. 29 Dec, 2023 1 commit
    • Kirill Smelkov's avatar
      nrarfcn: Fix behaviour on invalid input parameters · 8e606c64
      Kirill Smelkov authored
      Contrary to earfcn, where band can be automatically deduced from earfcn
      number because 4G bands never overlap, most functions in nrarfcn accept
      as input parameters both nr_arfcn and band, because 5G bands can and do
      overlap. As the result it is possible to invoke e.g. dl2ul with
      dl_nr_arfcn being outside of downlink spectrum of specified band.
      
      However in b8065120 I've made a thinko and handled such situation with
      simple assert which does not lead to useful error feedback from a user
      perspective, for example:
      
          In [2]: xnrarfcn.dl2ul(10000, 1)
          ---------------------------------------------------------------------------
          AssertionError                            Traceback (most recent call last)
          Cell In[2], line 1
          ----> 1 n.dl2ul(10000, 1)
      
          File ~/src/wendelin/xlte/nrarfcn.py:85, in dl2ul(dl_nr_arfcn, band)
               83 if dl_lo == 'N/A':
               84     raise AssertionError('band%r does not have downlink spectrum' % band)
          ---> 85 assert dl_lo <= dl_nr_arfcn <= dl_hi
               86 ul_lo, ul_hi = nr.get_nrarfcn_range(band, 'ul')
               87 if ul_lo == 'N/A':
      
          AssertionError:
      
      The issue here is that asserts can be used to only verify internal
      invariants, and that reported error does not provide details about which
      nrarfcn and band were used in the query.
      
      -> Fix this by providing details in the error reported to incorrect
      module usage, and by consistently raising ValueError for "invalid
      parameters" cases.
      
      The reported error for above example now becomes
      
          ValueError: band1: NR-ARFCN=10000 is outside of downlink spectrum
      8e606c64
  4. 05 Dec, 2023 1 commit
  5. 25 Oct, 2023 1 commit
    • Kirill Smelkov's avatar
      earfcn: New package to do computations with LTE bands, frequencies and EARFCN numbers · 6cb9d37f
      Kirill Smelkov authored
      Do a package which provides calculations like EARFCN -> frequency,
      EARFCN -> band info, and to convert DL/UL EARFCN in between each other.
      
      I was hoping to find something ready on the net, but could find only
      pypi.org/project/nrarfcn for 5G, while for LTE everything I found
      was of lesser quality and capability.
      
      -> So do it myself.
      
      See package documentation for API details.
      6cb9d37f
  6. 25 Jul, 2023 2 commits
    • Kirill Smelkov's avatar
      amari.xlog: Add support for arbitrary query options · bcfd82dd
      Kirill Smelkov authored
      Before this patch we were supporting only boolean option flags - with, for
      example, stats[rf] meaning stats query with {"rf": True} arguments. Now we add
      support for arbitrary types, so that it is possible to specify e.g. integer or
      string query options, as well as some boolean flag set to false.
      
      This should be good for generality.
      
      For backward compatibility the old way to implicitly specify "on" flags is
      continued to be supported.
      bcfd82dd
    • Kirill Smelkov's avatar
      amari.xlog: Test and polish str(LogSpec) · 70b4b71c
      Kirill Smelkov authored
      If the parsed period was '60s' we were printing it back as '60.0s' on str.
      Fix it by using %g insted of %s.
      70b4b71c
  7. 27 Apr, 2023 1 commit
  8. 20 Apr, 2023 1 commit
    • Kirill Smelkov's avatar
      kpi: Add way to compute aggregated counters + showcase this · bf96c767
      Kirill Smelkov authored
      - add Calc.cum to aggregate Measurements.
      
      - add ΣMeasurement type to represent result of this. It is very similar
        to Measurement, but every field comes accompanied with information
        about how much time there was no data for that field. In other words
        it is not all or nothing for NA in the result. For example a field
        might be present 90% of the time and NA only 10% of the time. We want to
        preserver knowledge about that 90% of valid values in the result. And we
        also want to know how much time there was no data.
      
      - amend kpidemo.py and kpidemo.ipynb to demonstrate this.
      bf96c767
  9. 18 Apr, 2023 3 commits
  10. 17 Apr, 2023 1 commit
    • Kirill Smelkov's avatar
      amari.xlog: Implement log rotation · a2c3afaa
      Kirill Smelkov authored
      Rotate output enb.xlog ourselves at sync points so that nothing is lost
      in the output (hello `logrotate copytruncate`) and so that we can emit
      pre- and post- logrotate syncs.
      
      Reuse logging's RotatingFileHandler and TimedRotatingFileHandler to
      implement actual rotation, but carefully wrap them in our writer
      classes so that we emit exactly the output we prepared explicitly
      without any headers prepended by logging, and that we explicitly control
      when rotation happens.
      
      /proposed-for-review-at !5
      a2c3afaa
  11. 28 Mar, 2023 1 commit
  12. 27 Mar, 2023 3 commits
  13. 22 Mar, 2023 6 commits
    • Kirill Smelkov's avatar
      amari.xlog: Sync, reverse reading, timestamps for eNB < 2022-12-01 · 67466ae5
      Kirill Smelkov authored
      Rework XLog protocol to come with periodic sync events that come from time to
      time so that xlog stream becomes self-synchronizing. Sync events should be
      useful for Wendelin to start reading xlog stream from any point, and to verify
      that the stream is ok by matching its content vs messages schedule coming in
      the syncs.
      
      Teach xlog.Reader to read streams in reverse order from end to start. This
      should be useful to look at tail of a log without reading it in full from the
      start.
      
      Teach xlog.Reader to reconstruct messages timestamps for xlog streams produced
      with Amarisoft releases < 2022-12-01. There messages do not have .utc field
      added in https://support.amarisoft.com/issues/21934 and come with only .time
      field that represent internal eNB time using clock originating at eNB startup.
      We combine message.time and δ(utc, enb.time) from sync to build message.timestamp .
      
      See individual patches for details and
      !3 for preliminary discussion.
      
      /reviewed-by @xavier_thompson
      /reviewed-on !4
      67466ae5
    • Kirill Smelkov's avatar
      amari.xlog: attach,sync += information about on-service time · 0c772eb4
      Kirill Smelkov authored
      We currently emit information about local time in events, and
      information about on-service time in messages. Events don't have
      information about on-service time and messages don't carry information
      about local time. That is mostly ok, since primary xlog setup is to run
      on the same machine, where eNB runs because on-service .utc correlates
      with .time in events.
      
      However for eNB < 2022-12-01 on-service time includes only .time field
      without .utc field with .time representing "time passed since when eNB
      was started". This way for enb.xlog streams generated on older systems
      it is not possible for xlog.Reader to know the absolute timestamps of
      read messages.
      
      To fix this we amend "attach" and "sync" events to carry both local and
      on-service times. This way xlog.Reader, after seeing e.g. "sync" with
      .time and only .srv_time without .srv_utc, should be able to
      correlate local and on-service clocks and to approximate srv_utc as
      
      	srv_utc' = srv_time' + (time - srv_time)
      
      where time and srv_time correspond to last synchronization, and
      srv_time' is what xlog.Reader retrieves for a further-read message in
      question.
      
      See !3 for related discussion.
      0c772eb4
    • Kirill Smelkov's avatar
      amari.xlog: Teach Reader to read xlog in reverse order from end to start · dbecc158
      Kirill Smelkov authored
      This functionality is useful to look at tail of a log without reading it
      in full from the start.
      dbecc158
    • Kirill Smelkov's avatar
      amari.xlog: Reader: tests: Verify yielded .pos · 515c1573
      Kirill Smelkov authored
      .pos verification should be there from xlog.Reader start in 0633d26f
      (amari.xlog += Reader)
      515c1573
    • Kirill Smelkov's avatar
      amari.xlog: Require sync to be present at least every 1000 records · b412d488
      Kirill Smelkov authored
      This way xlog.Reader can be sure that if it looked around in such a
      window and did not find a sync, then something is not good with the
      stream and it does not need to go beyond that limit looking around.
      
      This is a change of the protocol. But it is early days and existing logs
      - that we use in the demo, are all below 1000 lines limit, so they will
      continue to be loaded ok.
      
      No direct test for actual Loss Of Sync detection - this functionality is
      draft for now and should be improved later. However for no-LOS cases
      xlog.Reader is already covered with tests.
      b412d488
    • Kirill Smelkov's avatar
      amari.xlog: Unify start with sync · 9d9d20f3
      Kirill Smelkov authored
      Let's use "sync(reason=start)" instead of dedicated "start" event for
      uniformity. Periodic syncs are now "sync(reason=periodic)" and after
      logrotation support there will be also "pre-logrotate" and
      "post-logrotate" reasons. Emit "sync(reason=stop)" at xlog shutdown for
      uniformity and to make it more clear from looking at just enb.xlog about
      what is xlog state at the end.
      
      Stop requiring "start" to be present in the header - we will soon rework
      xlog reader to look around for nearby sync automatically so that reading
      could be started from any position in the stream.
      9d9d20f3
  14. 21 Mar, 2023 2 commits
    • Kirill Smelkov's avatar
      amari.xlog: Emit config_get after every sync(attached) instead of only after every attach · 67ece601
      Kirill Smelkov authored
      We emit config_get after every attach from the beginning of xlog in
      e0cc8a38 (amari.xlog: Initial draft). The reasoning here is that it is
      useful by default to know configuration of a service.
      
      In the previous patch we added sync events so that xlog stream becomes
      self-synchronizing. To continue that line it is now useful to have that
      config_get emitted after every such synchronization point instead of
      only after attaching to the service. That's what hereby patch does.
      
      As a bonus the code is reworked in a way that config_get setup is not
      hardcoded anymore and config_get periodicity now can be controlled by
      users via explicitly specifying config_get in the logspec.
      67ece601
    • Kirill Smelkov's avatar
      amari.xlog: Emit sync events periodically · 964f954a
      Kirill Smelkov authored
      So that xlog stream becomes self-synchronized and could be used even if
      we start reading it from some intermediate point instead of only from
      the beginning.
      
      We will need this in general - to be able to start reading long log not
      only from its beginning, and also in particular for Wendelin systems
      where logs are uploaded by Fluentd in chunks and some chunks could be
      potentially lost.
      
      Sync events are emitted always unconditionally with default sync
      interval being 10x the longest specified period. We also provide users a
      way to control sync periodicity via explicitly specifying
      "meta.sync/period" query in the logspec.
      
      See !3 (comment 175796) and
      further for related discussion.
      
      This is change of xlog protocol. But it is early days and the only
      direct consumer of xlog is amari.kpi which we adjust accordingly. So it
      should be ok.
      964f954a
  15. 17 Mar, 2023 2 commits
  16. 09 Mar, 2023 3 commits
    • Kirill Smelkov's avatar
      Draft support for E-UTRAN IP Throughput KPI · 2a016d48
      Kirill Smelkov authored
      The most interesting patches are
      
      - d102ffaa (drb: Start of the package)
      - 5bf7dc1c (amari.{drb,xlog}: Provide aggregated DRB statistics in the form of synthetic x.drb_stats message)
      - 499a7c1b (amari.kpi: Teach LogMeasure to handle x.drb_stats messages)
      - 2824f50d (kpi: Calc: Add support for E-UTRAN IP Throughput KPI)
      - 4b2c8c21 (demo/kpidemo.*: Add support for E-UTRAN IP Throughput KPI + demonstrate it in the notebook)
      
      The other patches introduce or adjust needed infrastructure. A byproduct
      of particular note is that kpi.Measurement now supports QCI.
      
      A demo might be seen in the last part of
      https://nbviewer.org/urls/lab.nexedi.com/kirr/xlte/raw/43aac33e/demo/kpidemo.ipynb
      
      And below we provide the overall overview of the implementation.
      
      Overview of E-UTRAN IP Throughput computation
      ---------------------------------------------
      
      Before we begin explaining how IP Throughput is computed, let's first refresh
      what it is and have a look at what is required to compute it reasonably.
      
      This KPI is defined in TS 32.450[1] and aggregates transmission volume and
      time over bursts of transmissions from an average UE point of view. It should be
      particularly noted that only the time, during which transmission is going on,
      should be accounted. For example if an UE receives 10KB over 4ms burst and the rest of
      the time there is no transmission to it during, say, 1 minute, the downlink IP
      Throughput for that UE over the minute is 20Mbit/s (= 8·10KB/4ms), not 1.3Kbit/s (= 8·10KB/60s).
      This KPI basically shows what would be the speed to e.g. download a response for
      HTTP request issued from a mobile.
      
      [1] https://www.etsi.org/deliver/etsi_ts/132400_132499/132450/16.00.00_60/ts_132450v160000p.pdf#page=13
      
      To compute IP Throughput we thus need to know Σ of transmitted amount
      of bytes, and Σ of the time of all transmission bursts.
      
      Σ of the bytes is relatively easy to get. eNB already provides close values in
      overall `stats` and in per-UE `ue_get[stats]` messages. However there is no
      anything readily available out-of-the box for Σ of bursts transmission time.
      Thus we need to measure the time of transmission bursts ourselves somehow.
      
      It turns out that with current state of things the only practical way to
      measure it to some degree is to poll eNB frequently with `ue_get[stats]` and
      estimate transmission time based on δ of `ue_get` timestamps.
      
      Let's see how frequently we need to poll to get to reasonably accuracy of resulting throughput.
      
      A common situation for HTTP requests issued via LTE is that response content
      downloading time takes only few milliseconds. For example I used chromium
      network profiler to access various sites via internet tethered from my phone
      and saw that for many requests response content downloading time was e.g. 4ms,
      5ms, 3.2ms, etc. The accuracy of measuring transmission time should be thus in
      the order of millisecond to cover that properly. It makes a real difference for
      reported throughput, if say a download sample with 10KB took 4ms, or it took
      e.g. "something under 100ms". In the first case we know that for that sample
      downlink throughput is 2500KB/s, while in the second case all we know is that
      downlink throughput is "higher than 100KB/s" - a 25 times difference and not
      certain. Similarly if we poll at 10ms rate we would get that throughput is "higher
      than 1000KB/s" - a 2.5 times difference from actual value. The accuracy of 1
      millisecond coincides with TTI time and with how downlink/uplink transmissions
      generally work in LTE.
      
      With the above the scheme to compute IP Throughput looks to be as
      follows: poll eNB at 1000Hz rate for `ue_get[stats]`, process retrieved
      information into per-UE and per-QCI streams, detect bursts on each UE/QCI pair,
      and aggregate `tx_bytes` and `tx_time` from every burst.
      
      It looks to be straightforward, but 1000Hz polling will likely create
      non-negligible additional load on the system and disturb eNB itself
      introducing much jitter and harming its latency requirements. That's probably
      why eNB actually rate-limits WebSocket requests not to go higher than 100Hz -
      the frequency 10 times less compared to what we need to get to reasonable
      accuracy for IP throughput.
      
      Fortunately there is additional information that provides a way to improve
      accuracy of measured `tx_time` even when polled every 10ms at 100Hz rate:
      that additional information is the number of transmitted transport blocks to/from
      an UE. If we know that during 10ms frame it was e.g. 4 transport blocks transmitted
      to the UE, that there were no retransmissions *and* that eNB is not congested, we can
      reasonably estimate that it was actually a 4ms transmission. And if eNB is
      congested we can still say that transmission time is somewhere in `[4ms, 10ms]`
      interval because transmitting each transport block takes 1 TTI. Even if
      imprecise that still provides some information that could be useful.
      
      Also 100Hz polling turns to be acceptable from performance point of view and
      does not disturb the system much. For example on the callbox machine the process,
      that issues polls, takes only about 3% of CPU load and only on one core, and
      the CPU usage of eNB does not practically change and its reported tx/rx latency
      does not change as well. For sure, there is some disturbance, but it appears to
      be small. To have a better idea of what rate of polling is possible, I've made
      an experiment with the poller accessing my own websocket echo server quickly
      implemented in python. Both the poller and the echo server are not optimized,
      but without rate-limiting they could go to 8000Hz frequency with reaching 100%
      CPU usage of one CPU core. That 8000Hz is 80x times more compared to 100Hz
      frequency actually allowed by eNB. This shows what kind of polling
      frequency limit the system can handle, if absolutely needed, and that 100Hz
      turns out to be not so high a frequency. Also the Linux 5.6 kernel, installed
      on the callbox from Fedora32, is configured with `CONFIG_HZ=1000`, which is
      likely helping here.
      
      Implementation overview
      ~~~~~~~~~~~~~~~~~~~~~~~
      
      The scheme to compute E-UTRAN IP Throughput is thus as follows: poll eNB at
      100Hz frequency for `ue_get[stats]` and retrieve information about per-UE/QCI
      streams and the number of transport blocks dl/ul-ed to the UE in question
      during that 10ms frame. Estimate `tx_time` taking into account
      the number of transmitted transport blocks. And estimate whether eNB is congested or
      not based on `dl_use_avg`/`ul_use_avg` taken from `stats`. For the latter we
      also need to poll for `stats` at 100Hz frequency and synchronize
      `ue_get[stats]` and `stats` requests in time so that they both cover the same
      time interval of particular frame.
      
      Then organize the polling process to provide aggregated statistics in the form of
      new `x.drb_stats` message, and teach `xamari xlog` to save that messages to
      `enb.xlog` together with `stats`. Then further adjust `amari.kpi.LogMeasure`
      and generic `kpi.Measurement` and `kpi.Calc` to handle DRB-related data.
      
      That is how it is implemented.
      
      The main part, that performs 100Hz polling and flow aggregation, is in amari/drb.py.
      There `Sampler` extracts bursts of data transmissions from stream of `ue_get[stats]`
      observations and `x_stats_srv` organizes whole 100Hz sampling process and provides
      aggregated `x.drb_stats` messages to `amari.xlog`.
      
      Even though the main idea is relatively straightforward, several aspects
      deserves to be noted:
      
      1. information about transmitted bytes and corresponding transmitted transport
         blocks is emitted by eNB not synchronized in time. The reason here is that,
         for example, for DL a block is transmitted via PDCCH+PDSCH during one TTI, and
         then the base station awaits HARQ ACK/NACK. That ACK/NACK comes later via
         PUCCH or PUSCH. The time window in between original transmission and
         reception of the ACK/NACK is 4 TTIs for FDD and 4-13 TTIs for TDD(*).
         And Amarisoft LTEENB updates counters for dl_total_bytes and dl_tx at
         different times:
      
             ue.erab.dl_total_bytes      - right after sending data on  PDCCH+PDSCH
             ue.cell.{dl_tx,dl_retx}     - after receiving ACK/NACK via PUCCH|PUSCH
      
         this way an update to dl_total_bytes might be seen in one frame (= 10·TTI),
         while corresponding update to dl_tx/dl_retx might be seen in either same, or
         next, or next-next frame.
      
         `Sampler` brings δ(tx_bytes) and #tx_tb in sync itself via `BitSync`.
      
      2. when we see multiple transmissions related to UE on different QCIs, we
         cannot directly use corresponding global number of transport blocks to estimate
         transmissions times because we do not know how eNB scheduler placed those
         transmissions onto resource map. So without additional information we can only
         estimate corresponding lower and upper bounds.
      
      3. for output stability and to avoid throughput being affected by partial fill
         of tail TTI of a burst, E-UTRAN IP Throughput is required to be computed
         without taking into account last TTI of every sample. We don't have that
         level of details since all we have is total amount of transmitted bytes in a
         burst and estimation of how long in time the burst is. Thus, once again, we
         can only provide an estimation so that resulting E-UTRAN IP
         Throughput uncertainty window cover the right value required by 3GPP standard.
      
      A curious reader might be interested to look at tests in `amari/drb_test.py` ,
      and at the whole changes that brought E-UTRAN IP Throughput alive.
      
      Limitations
      ~~~~~~~~~~~
      
      Current implementation has the following limitations:
      
      - we account whole PDCP instead of only IP traffic.
      - the KPI is computed with uncertainty window instead of being precise even when the
        connection to eNB is alive all the time. The shorter bursts are the more
        the uncertainty.
      - the implementation works correctly for FDD, but not for TDD. That's because
        BitSync currently supports only "next frame" case and support for "next-next
        frame" case is marked as TODO.
      - eNB `t` monitor command practically stops working and now only reports
        ``Warning, remote API ue_get (stats = true) pending...`` instead of reporting
        useful information. This is due to that contrary to `stats`, for `ue_get` eNB
        does not maintain per-connection state and uses global singleton counters.
      - the performance overhead might be more noticeable on machines less
        powerful compared to callbox.
      
      To address the limitations I plan to talk to Amarisoft about eNB improvements
      so that E-UTRAN IP Throughput could be computed precisely from DRB statistics
      directly provided by eNB itself.
      
      However it is still useful to have current implementation, even with all its
      limitations, because it already works today with existing eNB versions.
      
      Kirill
      2a016d48
    • Kirill Smelkov's avatar
      *: Cosmetics + minor · 43aac33e
      Kirill Smelkov authored
      Noticed while developing support for E-UTRAN IP Throughtput.
      43aac33e
    • Kirill Smelkov's avatar
      demo/kpidemo.*: Add support for E-UTRAN IP Throughput KPI + demonstrate it in the notebook · 4b2c8c21
      Kirill Smelkov authored
      Show how to compute that KPI, add corresponding plotting routines, and
      teach kpidemo.py to display both E-RAB Accessibility and E-UTRAN IP
      Throughput simultaneously in the same window.
      
      Add corresponding demonstration into demo notebook with data from
      throughput experiment showcasing several scenarious and how E-UTRAN IP
      Throughput implementation handles them.
      4b2c8c21
  17. 08 Mar, 2023 4 commits
    • Kirill Smelkov's avatar
      demo/kpidemo.*: Refactor commonly used bits into helper routines · 51785980
      Kirill Smelkov authored
      - move code to load amari.kpi.LogMeasure -> kpi.MeasurementLog into
        load_measurements(). We will need to use that when showcasing E-UTRAN
        IP Throughput KPI to load another enb.xlog dataset.
      - factor code to iterate over MeasurementLog and invoke kpi.Calc on each
        period into calc_each_period(). Same reason.
      - factor plotting code into helper routines located only in kpidemo.py.
        The notebook version now uses those routines by way of importing. The
        plotting code is not helping to understand the KPI computation
        pipeline usage, so it makes sense not to show it out of the box in the
        demo notebook.
      51785980
    • Kirill Smelkov's avatar
      t/udpflood: Test program to simulate transmission bursts · 1dc74b0c
      Kirill Smelkov authored
      It is useful to verify E-UTRAN IP Throughput KPI implementation, as
      that KPI is defined in terms of burst samples.
      1dc74b0c
    • Kirill Smelkov's avatar
      kpi: Calc: Add support for E-UTRAN IP Throughput KPI · 2824f50d
      Kirill Smelkov authored
      This patch provides the final building block for E-UTRAN IP Throughput KPI.
      It continues
      
          d102ffaa (drb: Start of the package)
          5bf7dc1c (amari.{drb,xlog}: Provide aggregated DRB statistics in the form of synthetic x.drb_stats message)
          499a7c1b (amari.kpi: Teach LogMeasure to handle x.drb_stats messages)
      
      Quoting those patches
      
          The scheme to compute E-UTRAN IP Throughput is thus as follows: poll eNB at
          100Hz frequency for `ue_get[stats]` and retrieve information about per-UE/QCI
          streams and the number of transport blocks dl/ul-ed to the UE in question
          during that 10ms frame. Estimate `tx_time` taking into account
          the number of transmitted transport blocks. And estimate whether eNB is congested or
          not based on `dl_use_avg`/`ul_use_avg` taken from `stats`. For the latter we
          also need to poll for `stats` at 100Hz frequency and synchronize
          `ue_get[stats]` and `stats` requests in time so that they both cover the same
          time interval of particular frame.
      
          Then organize the polling process to provide aggregated statistics in the form of
          new `x.drb_stats` message, and teach `xamari xlog` to save that messages to
          `enb.xlog` together with `stats`.
      
          Then further adjust `amari.kpi.LogMeasure` and generic `kpi.Measurement`
          and `kpi.Calc` to handle DRB-related data.						<-- NOTE
      
      So here we implement that last noted step:
      
      We add Calc.eutran_ip_throughput() whose implementation is relatively
      straightforward as the hard part is done by amari.drb and amari.kpi - in the
      Calc we basically need to only divide provided DRB.IPVolDl / DRB.IPTimeDl.
      2824f50d
    • Kirill Smelkov's avatar
      amari.kpi: Teach LogMeasure to handle x.drb_stats messages · 499a7c1b
      Kirill Smelkov authored
      This patch provides next building block for E-UTRAN IP Throughput KPI
      and continues
      
          d102ffaa (drb: Start of the package)
          5bf7dc1c (amari.{drb,xlog}: Provide aggregated DRB statistics in the form of synthetic x.drb_stats message)
      
      Quoting those patches
      
          The scheme to compute E-UTRAN IP Throughput is thus as follows: poll eNB at
          100Hz frequency for `ue_get[stats]` and retrieve information about per-UE/QCI
          streams and the number of transport blocks dl/ul-ed to the UE in question
          during that 10ms frame. Estimate `tx_time` taking into account
          the number of transmitted transport blocks. And estimate whether eNB is congested or
          not based on `dl_use_avg`/`ul_use_avg` taken from `stats`. For the latter we
          also need to poll for `stats` at 100Hz frequency and synchronize
          `ue_get[stats]` and `stats` requests in time so that they both cover the same
          time interval of particular frame.
      
          Then organize the polling process to provide aggregated statistics in the form of
          new `x.drb_stats` message, and teach `xamari xlog` to save that messages to
          `enb.xlog` together with `stats`.
      
          Then further adjust `amari.kpi.LogMeasure`						<-- NOTE
          and generic `kpi.Measurement` and `kpi.Calc` to handle DRB-related data.
      
      So here we implement the noted step:
      
      We teach LogMeasure to take x.drb_stats messages into account and update IP
      Throughput related fields in appropriate Measurement from x.drb_stats
      data.
      
      This process is relatively straightforward besides one place: for stable
      output E-UTRAN IP Throughput is required to be computed without taking
      into account last TTI of every sample. We don't have that level of
      details since all we have is total amount of transmitted bytes in a
      burst and estimation of how long in time the burst is. Thus we can only
      provide an estimation for the E-UTRAN IP Throughput as follows:
      
          DRB.IPVol and DRB.IPTime are collected to compute throughput.
      
          thp = ΣB*/ΣT*  where B* is tx'ed bytes in the sample without taking last tti into account
                         and   T* is time of tx also without taking that sample's tail tti.
      
          we only know ΣB (whole amount of tx), ΣT and ΣT* with some error.
      
          -> thp can be estimated to be inside the following interval:
      
                   ΣB            ΣB
                  ───── ≤ thp ≤ ─────           (1)
                  ΣT_hi         ΣT*_lo
      
          the upper layer in xlte.kpi will use the following formula for
          final throughput calculation:
      
                        DRB.IPVol
                  thp = ──────────              (2)
                        DRB.IPTime
      
          -> set DRB.IPTime and its error to mean and δ of ΣT_hi and ΣT*_lo
          so that (2) becomes (1).
      
      for this to work we also need to introduce new fields to Measurement
      that represent error of DRB.IPTime. The hope is that introduction is
      temporary and should be removed once we rework DRB stats to provide B*
      and T* directly.
      499a7c1b