trinity-1686a/garage
2
0
Fork 0
forked from Deuxfleurs/garage
Code Issues Pull requests 1 Projects Releases Wiki Activity

Added a first draft version of the algorithm and analysis for the non-strict mode.

Browse source
This commit is contained in:
Mendes 2022-08-19 21:21:41 +02:00
parent 7b2c065c82
commit 81083dd415
5 changed files with 120 additions and 49 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

15
doc/optimal_layout_report/optimal_layout.aux
View file

@ -7,26 +7,29 @@
\@writefile{toc}{\contentsline {subsection}{\numberline {2.1}Optimal assignment}{2}{}\protected@file@percent }
\newlabel{sec:opt_assign}{{2.1}{2}}
\@writefile{lof}{\contentsline {figure}{\numberline {1}{\ignorespaces On the left, the creation of a concrete assignment with the naive approach of repeating tokens. On the right, the zones containing the nodes.}}{4}{}\protected@file@percent }
\@writefile{lof}{\contentsline {figure}{\numberline {2}{\ignorespaces Flow problem to compute and optimal assignment.}}{4}{}\protected@file@percent }
\@writefile{toc}{\contentsline {subsection}{\numberline {2.2}Minimal transfer}{5}{}\protected@file@percent }
\newlabel{hyp:A}{{{H3A}}{5}}
\newlabel{hyp:B}{{{H3B}}{5}}
\@writefile{lof}{\contentsline {figure}{\numberline {2}{\ignorespaces Flow problem to compute and optimal assignment.}}{5}{}\protected@file@percent }
\@writefile{toc}{\contentsline {subsubsection}{\numberline {A)}Minimizing the zone discrepancy}{6}{}\protected@file@percent }
\@writefile{lof}{\contentsline {figure}{\numberline {3}{\ignorespaces On the left: the graph $G_T$ encoding an assignment to minimize the zone discrepancy. On the right: the graph $G_T$ encoding an assignment to minimize the node discrepancy.}}{7}{}\protected@file@percent }
\@writefile{toc}{\contentsline {subsubsection}{\numberline {B)}Minimizing the node discrepancy}{8}{}\protected@file@percent }
\@writefile{toc}{\contentsline {subsubsection}{\numberline {C)}Linear combination of both criteria}{8}{}\protected@file@percent }
\@writefile{toc}{\contentsline {subsubsection}{\numberline {C)}Linear combination of both criteria}{9}{}\protected@file@percent }
\@writefile{toc}{\contentsline {subsection}{\numberline {2.3}Algorithm}{9}{}\protected@file@percent }
\@writefile{loa}{\contentsline {algorithm}{\numberline {1}{\ignorespaces Optimal 3-strict assignment}}{9}{}\protected@file@percent }
\newlabel{alg:total}{{1}{9}}
\@writefile{toc}{\contentsline {section}{\numberline {3}TODO}{9}{}\protected@file@percent }
\@writefile{loa}{\contentsline {algorithm}{\numberline {2}{\ignorespaces Computation of the optimal utilization}}{10}{}\protected@file@percent }
\newlabel{alg:util}{{2}{10}}
\newlabel{lin:subutil}{{2}{10}}
\newlabel{lin:loopsub}{{3}{10}}
\newlabel{lin:findmin}{{4}{10}}
\@writefile{loa}{\contentsline {algorithm}{\numberline {3}{\ignorespaces Computation of a candidate assignment}}{10}{}\protected@file@percent }
\newlabel{alg:opt}{{3}{10}}
\@writefile{loa}{\contentsline {algorithm}{\numberline {3}{\ignorespaces Computation of a candidate assignment}}{11}{}\protected@file@percent }
\newlabel{alg:opt}{{3}{11}}
\@writefile{loa}{\contentsline {algorithm}{\numberline {4}{\ignorespaces Minimization of the number of transfers}}{11}{}\protected@file@percent }
\newlabel{alg:mini}{{4}{11}}
\newlabel{lin:repeat}{{3}{11}}
\gdef \@abspage@last{11}
\@writefile{toc}{\contentsline {section}{\numberline {3}Computation of a 3-non-strict assignment}{11}{}\protected@file@percent }
\@writefile{toc}{\contentsline {subsection}{\numberline {3.1}Choices of optimality}{11}{}\protected@file@percent }
\@writefile{toc}{\contentsline {subsection}{\numberline {3.2}Computation of a candidate assignment}{11}{}\protected@file@percent }
\@writefile{toc}{\contentsline {subsection}{\numberline {3.3}Maximal spread and minimal transfers}{12}{}\protected@file@percent }
\gdef \@abspage@last{13}

77
doc/optimal_layout_report/optimal_layout.log
View file

@ -1,4 +1,4 @@
This is pdfTeX, Version 3.14159265-2.6-1.40.21 (TeX Live 2020/Debian) (preloaded format=pdflatex 2022.6.23) 18 JUL 2022 22:33
This is pdfTeX, Version 3.14159265-2.6-1.40.21 (TeX Live 2020/Debian) (preloaded format=pdflatex 2022.6.23) 19 AUG 2022 21:20
entering extended mode
restricted \write18 enabled.
%&-line parsing enabled.
@ -228,35 +228,30 @@ LaTeX Font Info: Trying to load font information for U+msb on input line 17.
File: umsb.fd 2013/01/14 v3.01 AMS symbols B
) [1
{/var/lib/texmf/fonts/map/pdftex/updmap/pdftex.map}] [2]
<figures/naive.pdf, id=23, 248.75592pt x 93.43271pt>
{/var/lib/texmf/fonts/map/pdftex/updmap/pdftex.map}] [2] [3]
<figures/naive.pdf, id=28, 248.75592pt x 93.43271pt>
File: figures/naive.pdf Graphic file (type pdf)
<use figures/naive.pdf>
Package pdftex.def Info: figures/naive.pdf used on input line 117.
Package pdftex.def Info: figures/naive.pdf used on input line 121.
(pdftex.def) Requested size: 310.4979pt x 116.6252pt.
[3]
<figures/flow.pdf, id=29, 263.10938pt x 133.389pt>
File: figures/flow.pdf Graphic file (type pdf)
<use figures/flow.pdf>
Package pdftex.def Info: figures/flow.pdf used on input line 136.
Package pdftex.def Info: figures/flow.pdf used on input line 140.
(pdftex.def) Requested size: 207.0021pt x 104.94873pt.
[4 <./figures/naive.pdf> <./figures/flow.pdf
pdfTeX warning: /usr/bin/pdflatex (file ./figures/flow.pdf): PDF inclusion: mul
tiple pdfs with page group included in a single page
>] [5]
<figures/mini_zone.pdf, id=40, 106.06543pt x 133.44284pt>
[4 <./figures/naive.pdf>] [5 <./figures/flow.pdf>] [6]
<figures/mini_zone.pdf, id=44, 106.06543pt x 133.44284pt>
File: figures/mini_zone.pdf Graphic file (type pdf)
<use figures/mini_zone.pdf>
Package pdftex.def Info: figures/mini_zone.pdf used on input line 221.
Package pdftex.def Info: figures/mini_zone.pdf used on input line 225.
(pdftex.def) Requested size: 110.39873pt x 138.8974pt.
<figures/mini_node.pdf, id=41, 183.67638pt x 190.3113pt>
<figures/mini_node.pdf, id=45, 183.67638pt x 190.3113pt>
File: figures/mini_node.pdf Graphic file (type pdf)
<use figures/mini_node.pdf>
Package pdftex.def Info: figures/mini_node.pdf used on input line 225.
Package pdftex.def Info: figures/mini_node.pdf used on input line 229.
(pdftex.def) Requested size: 151.8014pt x 157.28752pt.
[6]
Overfull \hbox (6.52959pt too wide) in paragraph at lines 239--240
Overfull \hbox (6.52959pt too wide) in paragraph at lines 243--244
[]\OT1/cmr/m/n/10 Assume that their ex-ist some as-sign-ment $\OML/cmm/m/it/10
T[]$ \OT1/cmr/m/n/10 with the same uti-liza-tion $(\OML/cmm/m/it/10 n[]\OT1/cmr
/m/n/10 )[]$.
@ -266,38 +261,38 @@ T[]$ \OT1/cmr/m/n/10 with the same uti-liza-tion $(\OML/cmm/m/it/10 n[]\OT1/cmr
pdfTeX warning: /usr/bin/pdflatex (file ./figures/mini_node.pdf): PDF inclusion
: multiple pdfs with page group included in a single page
>] [8] [9] [10] [11] (./optimal_layout.aux) )
>] [8] [9] [10] [11] [12] [13] (./optimal_layout.aux) )
Here is how much of TeX's memory you used:
3544 strings out of 481176
47263 string characters out of 5914226
339215 words of memory out of 5000000
336215 words of memory out of 5000000
20458 multiletter control sequences out of 15000+600000
413592 words of font info for 65 fonts, out of 8000000 for 9000
59 hyphenation exceptions out of 8191
68i,12n,74p,880b,308s stack positions out of 5000i,500n,10000p,200000b,80000s
</home/netko/.texlive2020/texmf-v
ar/fonts/pk/ljfour/jknappen/ec/tcrm1000.600pk></usr/share/texlive/texmf-dist/fo
nts/type1/public/amsfonts/cm/cmbx10.pfb></usr/share/texlive/texmf-dist/fonts/ty
pe1/public/amsfonts/cm/cmbx12.pfb></usr/share/texlive/texmf-dist/fonts/type1/pu
blic/amsfonts/cm/cmcsc10.pfb></usr/share/texlive/texmf-dist/fonts/type1/public/
amsfonts/cm/cmex10.pfb></usr/share/texlive/texmf-dist/fonts/type1/public/amsfon
ts/cm/cmmi10.pfb></usr/share/texlive/texmf-dist/fonts/type1/public/amsfonts/cm/
cmmi5.pfb></usr/share/texlive/texmf-dist/fonts/type1/public/amsfonts/cm/cmmi7.p
fb></usr/share/texlive/texmf-dist/fonts/type1/public/amsfonts/cm/cmr10.pfb></us
r/share/texlive/texmf-dist/fonts/type1/public/amsfonts/cm/cmr12.pfb></usr/share
/texlive/texmf-dist/fonts/type1/public/amsfonts/cm/cmr17.pfb></usr/share/texliv
e/texmf-dist/fonts/type1/public/amsfonts/cm/cmr5.pfb></usr/share/texlive/texmf-
dist/fonts/type1/public/amsfonts/cm/cmr7.pfb></usr/share/texlive/texmf-dist/fon
ts/type1/public/amsfonts/cm/cmr8.pfb></usr/share/texlive/texmf-dist/fonts/type1
/public/amsfonts/cm/cmsy10.pfb></usr/share/texlive/texmf-dist/fonts/type1/publi
c/amsfonts/cm/cmsy5.pfb></usr/share/texlive/texmf-dist/fonts/type1/public/amsfo
nts/cm/cmsy7.pfb></usr/share/texlive/texmf-dist/fonts/type1/public/amsfonts/cm/
cmti10.pfb></usr/share/texlive/texmf-dist/fonts/type1/public/amsfonts/symbols/m
sbm10.pfb>
Output written on optimal_layout.pdf (11 pages, 279062 bytes).
</home/netko/.texlive20
20/texmf-var/fonts/pk/ljfour/jknappen/ec/tcrm1000.600pk></usr/share/texlive/tex
mf-dist/fonts/type1/public/amsfonts/cm/cmbx10.pfb></usr/share/texlive/texmf-dis
t/fonts/type1/public/amsfonts/cm/cmbx12.pfb></usr/share/texlive/texmf-dist/font
s/type1/public/amsfonts/cm/cmcsc10.pfb></usr/share/texlive/texmf-dist/fonts/typ
e1/public/amsfonts/cm/cmex10.pfb></usr/share/texlive/texmf-dist/fonts/type1/pub
lic/amsfonts/cm/cmmi10.pfb></usr/share/texlive/texmf-dist/fonts/type1/public/am
sfonts/cm/cmmi5.pfb></usr/share/texlive/texmf-dist/fonts/type1/public/amsfonts/
cm/cmmi7.pfb></usr/share/texlive/texmf-dist/fonts/type1/public/amsfonts/cm/cmr1
0.pfb></usr/share/texlive/texmf-dist/fonts/type1/public/amsfonts/cm/cmr12.pfb><
/usr/share/texlive/texmf-dist/fonts/type1/public/amsfonts/cm/cmr17.pfb></usr/sh
are/texlive/texmf-dist/fonts/type1/public/amsfonts/cm/cmr5.pfb></usr/share/texl
ive/texmf-dist/fonts/type1/public/amsfonts/cm/cmr7.pfb></usr/share/texlive/texm
f-dist/fonts/type1/public/amsfonts/cm/cmr8.pfb></usr/share/texlive/texmf-dist/f
onts/type1/public/amsfonts/cm/cmsy10.pfb></usr/share/texlive/texmf-dist/fonts/t
ype1/public/amsfonts/cm/cmsy5.pfb></usr/share/texlive/texmf-dist/fonts/type1/pu
blic/amsfonts/cm/cmsy7.pfb></usr/share/texlive/texmf-dist/fonts/type1/public/am
sfonts/cm/cmti10.pfb></usr/share/texlive/texmf-dist/fonts/type1/public/amsfonts
/symbols/msbm10.pfb>
Output written on optimal_layout.pdf (13 pages, 289460 bytes).
PDF statistics:
127 PDF objects out of 1000 (max. 8388607)
90 compressed objects within 1 object stream
135 PDF objects out of 1000 (max. 8388607)
96 compressed objects within 1 object stream
0 named destinations out of 1000 (max. 500000)
21 words of extra memory for PDF output out of 10000 (max. 10000000)

BIN
doc/optimal_layout_report/optimal_layout.pdf
View file

Binary file not shown.

BIN
doc/optimal_layout_report/optimal_layout.synctex.gz
View file

Binary file not shown.

77
doc/optimal_layout_report/optimal_layout.tex
View file

@ -54,6 +54,10 @@ For now, in the following, we ask the following redundancy constraint:
\textbf{Mode 3-strict:} every partition needs to be assignated to three nodes belonging to three different zones.
\textbf{Mode 3:} every partition needs to be assignated to three nodes. We try to spread the three nodes over different zones as much as possible.
\textbf{Remark: (TODO):} The algorithms below directly adapt to a redundancy of $r$ instead of 3.
\section{Properties of an optimal 3-strict assignment}
\subsection{Optimal assignment}
@ -384,11 +388,80 @@ The graph $G_T$ has $O(N)$ vertices and $O(N\times \#Z)$ edges under assumption
\end{algorithmic}
\end{algorithm}
\newpage
\section{Computation of a 3-non-strict assignment}
\subsection{Choices of optimality}
In this mode, we primarily want to store every partition on three nodes, and only secondarily try to spread the nodes among different zone. So we make the choice of not taking the zone repartition in the criterion of optimality.
We try to maximize $s^*$ defined in \eqref{eq:optimal}. So we can compute the optimal utilizations $(n_v)_{v\in V}$ with the only constraint that $n_v \le N$ for every node $v$. As in the previous section, we start with a sub-utilization proportional to $c_v$ (and capped at $N$), and we iteratively increase the $\hat{n}_v$ that is less than $N$ and maximizes the quantity $c_v/(\hat{n}_v+1)$, until the total sum is $3N$.
\subsection{Computation of a candidate assignment}
To compute a candidate assignment (that does not optimize zone spreading nor distance to a previous assignment yet), we can use the folowing flow problem.
Define the oriented weighted graph $(X,E)$. The set of vertices $X$ contains the source $\mathbf{s}$, the sink $\mathbf{t}$, vertices
$\mathbf{x}_p, \mathbf{u}^+_p, \mathbf{u}^-_p$ for every partition $p$, vertices $\mathbf{y}_{p,z}$ for every partition $p$ and zone $z$, and vertices $\mathbf{z}_v$ for every node $v$.
The set of edges is composed of the following arcs:
\begin{itemize}
\item ($\mathbf{s}$,$\mathbf{x}_p$, 3) for every partition $p$;
\item ($\mathbf{x}_p$,$\mathbf{u}^+_p$, 3) for every partition $p$;
\item ($\mathbf{x}_p$,$\mathbf{u}^-_p$, 2) for every partition $p$;
\item ($\mathbf{u}^+_p$,$\mathbf{y}_{p,z}$, 1) for every partition $p$ and zone $z$;
\item ($\mathbf{u}^-_p$,$\mathbf{y}_{p,z}$, 2) for every partition $p$ and zone $z$;
\item ($\mathbf{y}_{p,z}$,$\mathbf{z}_v$, 1) for every partition $p$, zone $z$ and node $v\in z$;
\item ($\mathbf{z}_v$, $\mathbf{t}$, $n_v$) for every node $v$;
\end{itemize}
One can check that any maximal flow in this graph corresponds to an assignment of partitions to nodes. In such a flow, all the arcs from $\mathbf{s}$ and to $\mathbf{t}$ are saturated. The arc from $\mathbf{y}_{p,z}$ to $\mathbf{z}_v$ is saturated if and only if $p$ is associated to~$v$.
Finally the flow from $\mathbf{x}_p$ to $\mathbf{y}_{p,z}$ can go either through $\mathbf{u}^+_p$ or $\mathbf{u}^-_p$.
\section{TODO}
- reunion deux fleurs : autres modes, autres contraintes
\subsection{Maximal spread and minimal transfers}
Notice that if the arc $\mathbf{u}_p^+\mathbf{y}_{p,z}$ is not saturated but there is some flow in $\mathbf{u}_p^-\mathbf{y}_{p,z}$, then it is possible to transfer a unit of flow from the path $\mathbf{x}_p\mathbf{u}_p^-\mathbf{y}_{p,z}$ to the path $\mathbf{x}_p\mathbf{u}_p^+\mathbf{y}_{p,z}$. So we can always find an equivalent maximal flow $f^*$ that uses the path through $\mathbf{u}_p^-$ only if the path through $\mathbf{u}_p^+$ is saturated.
We will use this fact to consider the amount of flow going through the vertices $\mathbf{u}^+$ as a measure of how well the partitions are spread over nodes belonging to different zones. If the partition $p$ is associated to 3 different zones, then a flow of 3 will cross $\mathbf{u}_p^+$ in $f^*$ (i.e. a flow of 0 will cross $\mathbf{u}_p^+$). If $p$ is associated to two zones, a flow of $2$ will cross $\mathbf{u}_p^+$. If $p$ is associated to a single zone, a flow of $1$ will cross $\mathbf{u}_p^+$.
Let $N_1, N_2, N_3$ be the number of partitions associated to respectively 1,2 and 3 distinct zones. We will optimize a linear combination of these variables using the discovery of positively weighted circuits in a graph.
At the same step, we will also optimize the distance to a previous assignment $T'$. Let $\alpha> \beta> \gamma \ge 0$ be three parameters.
Given the flow $f$, let $G_f=(X',E_f)$ be the multi-graph where $X' = X\setminus\{\mathbf{s},\mathbf{t}\}$. The set $E_f$ is composed of the arcs:
\begin{itemize}
\item As many arcs from $(\mathbf{x}_p, \mathbf{u}^+_p,\alpha), (\mathbf{x}_p, \mathbf{u}^+_p,\beta), (\mathbf{x}_p, \mathbf{u}^+_p,\gamma)$ (selected in this order) as there is flow crossing $\mathbf{u}^+_p$ in $f$;
\item As many arcs from $(\mathbf{u}^+_p, \mathbf{x}_p,-\gamma), (\mathbf{u}^+_p, \mathbf{x}_p,-\beta), (\mathbf{u}^+_p, \mathbf{x}_p,-\alpha)$ (selected in this order) as there is flow crossing $\mathbf{u}^-_p$ in $f$;
\item As many copies of $(\mathbf{x}_p, \mathbf{u}^-_p,0)$ as there is flow through $\mathbf{u}^-_p$;
\item As many copies of $(\mathbf{u}^-_p,\mathbf{x}_p,0)$ so that the number of arcs between these two vertices is 2;
\item $(\mathbf{u}^+_p,\mathbf{y}_{p,z}, 0)$ if the flow between these vertices is 1, and the opposite arc otherwise;
\item as many copies of $(\mathbf{u}^-_p,\mathbf{y}_{p,z}, 0)$ as the flow between these vertices, and as many copies of the opposite arc as 2~$-$~the flow;
\item $(\mathbf{y}_{p,z},\mathbf{z}_v, \pm1)$ if it is saturated in $f$, with $+1$ if $v\in T'_p$ and $-1$ otherwise;
\item $(\mathbf{z}_v,\mathbf{y}_{p,z}, \pm1)$ if it is not saturated in $f$, with $+1$ if $v\notin T'_p$ and $-1$ otherwise.
\end{itemize}
To summarize, arcs are oriented left to right if they correspond to a presence of flow in $f$, and right to left if they correspond to an absence of flow. They are positively weighted if we want them to stay at their current state, and negatively if we want them to switch. Let us compute the weight of such graph.
\begin{multline*}
w(G_f) = \sum_{e\in E_f} w(e_f) \\
=
(\alpha - \beta -\gamma) N_1 + (\alpha +\beta - \gamma) N_2 + (\alpha+\beta+\gamma) N_3
\\ +
\#V\times N - 4 \sum_p 3-\#(T_p\cap T'_p) \\
=(\#V-12+\alpha-\beta-\gamma)\times N + 4Q_V + 2\beta N_2 + 2(\beta+\gamma) N_3 \\
\end{multline*}
As for the mode 3-strict, one can check that the difference of two such graphs corresponding to the same $(n_v)$ is always eulerian. Hence we can navigate in this class with the same greedy algorithm that discovers positive cycles and flips them.
The function that we optimize is
$$
2Q_V + \beta N_2 + (\beta+\gamma) N_3.
$$
The choice of parameters $\beta$ and $\gamma$ should be lead by the following question: For $\beta$, where to put the tradeoff between zone dispersion and distance to the previous configuration? For $\gamma$, do we prefer to have more partitions spread between 2 zones, or have less between at least 2 zones but more between 3 zones.
The quantity $Q_V$ varies between $0$ and $3N$, it should be of order $N$. The quantity $N_2+N_3$ should also be of order $N$ (it is exactly $N$ in the strict mode). So the two terms of the function are comparable.
\end{document}